From rustompmody at gmail.com Sun Feb 1 00:36:18 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 31 Jan 2015 21:36:18 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sunday, February 1, 2015 at 10:15:13 AM UTC+5:30, Ethan Furman wrote: > On 01/31/2015 07:16 PM, Steven D'Aprano wrote: > > > > But by default, Python will fallback on __repr__ if __str__ doesn't exist, > > or __str__ if __repr__ doesn't exist, or both. Or something. (I always > > forget what the rules are exactly.) > > If __str__ is missing, __repr__ is called. > > If __repr__ is missing, object.__repr__ (or some intermediate base class' __repr__) is called. > > -- > ~Ethan~ The other day I was taking a class in which I was showing - introspection for discovering -- help, type, dir etc at the repl - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) And a student asked me the diff between dir([]) and [].__dir__() I didnt know what to say... Now surely the amount of python I dont know is significantly larger than what I know Still it would be nice to have surface-syntax ?? dunder-magic more systematically documented From no.email at nospam.invalid Sun Feb 1 01:12:10 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 31 Jan 2015 22:12:10 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> Message-ID: <8761bm40ud.fsf@jester.gateway.sonic.net> Steven D'Aprano writes: > Some degree of weakness in a type system is not necessarily bad. Even the > strongest of languages usually allow a few exceptions, such as numeric > coercions. Haskell doesn't have automatic coercions of any sort. You have to call a conversion function if you want to turn an Int into an Integer. > I've never come across a language that has pointers which insists on > having a separate Nil pointer for ever pointer type Haskell's idiomatic substitute for a null pointer is a Nothing value (like Python's None) and there's a separate one for every type. The FFI offers actual pointers (Foreign.Ptr) and there is a separate nullPtr for every type. > the compiler will allow Nil to be used for any pointer type. Anything > else would be impractical. It's completely practical: polymorphism and type inference get you the value you want with usually no effort on your part. > What if you add two empty objects? > js> {} + {} OMG, javascript is worse than I thought > https://www.destroyallsoftware.com/talks/wat Can't view, needs flash. :( Try this instead (NFSW): https://www.youtube.com/watch?v=FJ7QsEytQq4 From no.email at nospam.invalid Sun Feb 1 01:26:25 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 31 Jan 2015 22:26:25 -0800 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> Message-ID: <871tma406m.fsf@jester.gateway.sonic.net> Marko Rauhamaa writes: > The guiding principle in C++ language development is to take static > type safety to the extreme. Heh, try Ada. > Stroustrup apparently has never had to deal with callbacks; his thick > books never made a mention of them last time I checked. C++ has function pointers just like C, but more idiomatically you'd pass a class instance and the library would invoke some method on it. There is also Boost::Coroutine which can get rid of the need for callbacks in some situations. > Esthetically, I'm most impressed with Scheme. One day it might give > Python a run for its money. It's in a small and shrinking niche, unfortunately. From frank at chagford.com Sun Feb 1 01:45:54 2015 From: frank at chagford.com (Frank Millman) Date: Sun, 1 Feb 2015 08:45:54 +0200 Subject: CSV and number formats Message-ID: Hi all I downloaded some bank statements in CSV format with a view to providing an automated bank reconciliation feature for my accounting software. One of them shows the opening balance in an unusual format. Most transaction amounts are in the format '-00000031.23' or '+00000024.58' This can easily be parsed using decimal.Decimal(). If the opening balance is positive, it appears as '+00000021.45' If it is negative, it appears as '+00000-21.45' Predictably, decimal.Decimal does not like this. Is this a recognised format, and is there a standard way of parsing it? If not, I will have to special-case it, but I would prefer to avoid that if possible. Thanks Frank Millman From jeanpierreda at gmail.com Sun Feb 1 01:54:44 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sat, 31 Jan 2015 22:54:44 -0800 Subject: Python is DOOMED! Again! In-Reply-To: <8761bm40ud.fsf@jester.gateway.sonic.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> Message-ID: Sorry, sort of responding to both of you. On Sat, Jan 31, 2015 at 10:12 PM, Paul Rubin wrote: > Steven D'Aprano writes: >> Some degree of weakness in a type system is not necessarily bad. Even the >> strongest of languages usually allow a few exceptions, such as numeric >> coercions. > > Haskell doesn't have automatic coercions of any sort. You have to call > a conversion function if you want to turn an Int into an Integer. Yeah. In fact, it isn't very compatible with the ML/Haskell type system to automatically convert, because it does weird things to type inference and type unification. So this is common in that language family. That said, Haskell (and the rest) do have a sort of type coercion, of literals at compile time (e.g. 3 can be an Integer or a Double depending on how you use it.) BTW it's weird that in this thread, and in the programmer community at large, int->string is considered worse than int->float, when the former is predictable and reversible, while the latter is lossy and can cause subtle bugs. Although at least we don't have ten+ types with sixty different spellings which change from platform to platform, and all of which automatically coerce despite massive and outrageous differences in representable values. (Hello, C.) >> I've never come across a language that has pointers which insists on >> having a separate Nil pointer for ever pointer type > > Haskell's idiomatic substitute for a null pointer is a Nothing value > (like Python's None) and there's a separate one for every type. The FFI > offers actual pointers (Foreign.Ptr) and there is a separate nullPtr > for every type. For that matter, how is this (first part) different from, say, Java? It's really only dynamically typed languages that have a single null value of a single type. Maybe I misunderstand the original statement. -- Devin From breamoreboy at yahoo.co.uk Sun Feb 1 02:23:16 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 01 Feb 2015 07:23:16 +0000 Subject: CSV and number formats In-Reply-To: References: Message-ID: On 01/02/2015 06:45, Frank Millman wrote: > Hi all > > I downloaded some bank statements in CSV format with a view to providing an > automated bank reconciliation feature for my accounting software. > > One of them shows the opening balance in an unusual format. > > Most transaction amounts are in the format '-00000031.23' or '+00000024.58' > > This can easily be parsed using decimal.Decimal(). > > If the opening balance is positive, it appears as '+00000021.45' > > If it is negative, it appears as '+00000-21.45' > > Predictably, decimal.Decimal does not like this. > > Is this a recognised format, and is there a standard way of parsing it? If > not, I will have to special-case it, but I would prefer to avoid that if > possible. > > Thanks > > Frank Millman > I've never seen anything like it and have to believe that this is a bug, although I'd guess that the bank in question will claim it's a feature. You'll surely have to work around it whilst waiting for a reply to the "WTF do you think you're playing at?" question. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marko at pacujo.net Sun Feb 1 02:38:54 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 01 Feb 2015 09:38:54 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> Message-ID: <87a90yf5dd.fsf@elektro.pacujo.net> Rustom Mody : > There is a real conflict between the logician's goal and the > educator's. The logician wants to minimize the variety of ideas, > and doesn't mind a long, thin path. The educator (rightly) wants > to make the paths short and doesn't mind?in fact, > prefers?connections to many other ideas. And he cares almost not > at all about the directions of the links. The natural language has a rigorous grammar plus a lexicon that includes a number of idioms. Nobody has so far been able to codify a natural language completely because the rigorous grammar consists of maybe 10,000 rules. Perl came out of the needs of natural language translation. Perl's novel idea was to make a programming language like a natural language (yes, there had been Cobol). So Perl's language description evades a complete, rigorous description, but it has numerous handy idioms for many common situations. Python showed that while interesting and amusing, Perl's way leads to unnecessary clutter. Why make things complicated when simple will serve everybody's needs best. Marko From marko at pacujo.net Sun Feb 1 02:58:32 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 01 Feb 2015 09:58:32 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> Message-ID: <8761bmf4gn.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> Stroustrup apparently has never had to deal with callbacks; his thick >> books never made a mention of them last time I checked. > > C++ has function pointers just like C, Et tu, Brute! C's callbacks always use a void pointer for the "self reference." In C, I can use void pointers and type casts idiomatically. In C++, type casts are apostasy. Qt gave up on C++ when it comes to callbacks ("signals") and went for an apocryphal metacompiler. > but more idiomatically you'd pass a class instance and the library > would invoke some method on it. Yes, that's what I ended up doing, defining a class for each callback type: struct ButtonPushListener { virtual void buttonPush(int x, int y) = 0; }; (Yes, "struct" and not "class"! Why?) Typing all of that in was quite a chore. All because Stroustrup didn't think of delegates (Delphi, C#, Python). C++'s method pointers are ridiculous and useless, they should have been defined as delegates. > There is also Boost::Coroutine which can get rid of the need for > callbacks in some situations. Boost is the world's biggest fig leaf. Marko From ethan at stoneleaf.us Sun Feb 1 03:12:10 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 01 Feb 2015 00:12:10 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54CDDFDA.8030505@stoneleaf.us> On 01/31/2015 09:36 PM, Rustom Mody wrote: > > And a student asked me the diff between > dir([]) > and > [].__dir__() > > I didnt know what to say... > Now surely the amount of python I dont know is significantly larger than what I know > Still it would be nice to have surface-syntax ?? dunder-magic more > systematically documented I don't have a complete answer for you, but I can say this: In simple cases (such as __len__) there is little difference between calling the surface operator and the dunder version (the error message differs in this case). In more complex cases (such as __add__) using the surface syntax (+) buys lots of extras: - if one operand is a subclass of the other, calling its __add__ or __radd__ method as appropriate - if the first operand returns NotImplemented, calling the other operand's __radd__ to see if that works - possibly others that I don't recall at the moment Basically, unless you're programming at the system (or class internals) level, don't call dunder methods directly. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From auriocus at gmx.de Sun Feb 1 03:14:10 2015 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 01 Feb 2015 09:14:10 +0100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <8761bmf4gn.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> Message-ID: Am 01.02.15 um 08:58 schrieb Marko Rauhamaa: > Paul Rubin : > >> Marko Rauhamaa writes: >>> Stroustrup apparently has never had to deal with callbacks; his thick >>> books never made a mention of them last time I checked. >> >> C++ has function pointers just like C, > > Et tu, Brute! > > C's callbacks always use a void pointer for the "self reference." In C, > I can use void pointers and type casts idiomatically. In C++, type casts > are apostasy. > > Qt gave up on C++ when it comes to callbacks ("signals") and went for an > apocryphal metacompiler. Yes, but only because C++ compilers were not good enough when QT came out, and later is was too late to change it to a templated system. Lookup libsigc++ http://libsigc.sourceforge.net/ which does the same using standard C++ and http://qt-project.org/doc/qt-4.8/templates.html for a reasoning of QT. In C++11 (supported by MSVC, g++, clang) there re also lambda expressions Christian From ethan at stoneleaf.us Sun Feb 1 03:22:21 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 01 Feb 2015 00:22:21 -0800 Subject: CSV and number formats In-Reply-To: References: Message-ID: <54CDE23D.5020402@stoneleaf.us> On 01/31/2015 11:23 PM, Mark Lawrence wrote: > On 01/02/2015 06:45, Frank Millman wrote: >> >> >> Most transaction amounts are in the format '-00000031.23' or '+00000024.58' >> >> This can easily be parsed using decimal.Decimal(). >> >> If the opening balance is positive, it appears as '+00000021.45' >> >> If it is negative, it appears as '+00000-21.45' 1) I have never seen that format before. 2) Regular transactions (non-opening balance) appear normally. 3) That's a bug! 4) Easiest workaround is probably to lstrip('+0'). -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From __peter__ at web.de Sun Feb 1 05:11:31 2015 From: __peter__ at web.de (Peter Otten) Date: Sun, 01 Feb 2015 11:11:31 +0100 Subject: parsing tree from excel sheet References: Message-ID: alb wrote: >> But wait, "".join() only accepts strings so let's change >> >>yield [node] >> >> to >>yield [node.name] # str(node) would also work > > Again my question, why not simply yield node.name? I've been conditioned to build a string from many substrings like so >>> parts = ["foo", "bar", "baz"] >>> text = "".join(parts) >>> text 'foobarbaz' instead of >>> text = "" >>> for part in parts: ... text += part ... >>> text 'foobarbaz' mostly because ""join(...) was initially advertised as being faster than +=. For nested generators or functions this translates into >>> def outer_list(): ... return ["foo"] + inner_list() ... >>> def inner_list(): ... return ["bar"] ... >>> "".join(outer_list()) 'foobar' instead of the obvious >>> def outer(): ... return "foo" + inner() ... >>> def inner(): ... return "bar" ... >>> outer() 'foobar' Here the list-based approach may build many intermediate throwaway-lists, so it's most likely less efficient than direct string concatenation. In return it gives you flexibility: >>> "/".join(outer_list()) 'foo/bar' >>> "-->".join(outer_list()) 'foo-->bar' You'd have to pass the separator as an argument to outer/inner() to achieve this with the seemingly simpler approach. But there's more: You can reverse the order, >>> ":".join(reversed(outer_list())) 'bar:foo' treat the innermost string differently, translate each part into a different language, clip common levels, etc. Your options are unlimited. I never understood why the file system API operates on a single string... From jesseporeddu at aim.com Sun Feb 1 08:29:39 2015 From: jesseporeddu at aim.com (VOGLIAMO RENZUSCONI IN GALERA!) Date: Sun, 1 Feb 2015 05:29:39 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From skip.montanaro at gmail.com Sun Feb 1 08:50:22 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 07:50:22 -0600 Subject: CSV and number formats In-Reply-To: References: Message-ID: On Sun, Feb 1, 2015 at 12:45 AM, Frank Millman wrote: > Is this a recognised format, and is there a standard way of parsing it? If > not, I will have to special-case it, but I would prefer to avoid that if > possible. Doesn't look "standard" to me in any fashion. You shouldn't need to special case it though, just s = re.sub("^[+]0*", "", s) e.g., >>> s = '+00000-21.45' >>> t = '+0000021.45' >>> re.sub("^[+]0*", "", s) '-21.45' >>> re.sub("^[+]0*", "", t) '21.45' That should work on all numbers, not just the weird one. Unless you have bazillions of numbers, it shouldn't be a performance hit either. Skip From ginobusciarello at outlook.com Sun Feb 1 08:56:37 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 1 Feb 2015 05:56:37 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <86bb78fb-fc03-46b2-8ad9-e7f2157d85d1@googlegroups.com> -- redacted -- From ginobusciarello at outlook.com Sun Feb 1 08:57:09 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 1 Feb 2015 05:57:09 -0800 (PST) Subject: -- redacted -- In-Reply-To: <35861325-94c1-418c-ab89-e4ea967431da@googlegroups.com> References: <35861325-94c1-418c-ab89-e4ea967431da@googlegroups.com> Message-ID: <597f1788-60a1-4fe5-9b28-2e3c452fca71@googlegroups.com> -- redacted -- From ginobusciarello at outlook.com Sun Feb 1 08:57:24 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 1 Feb 2015 05:57:24 -0800 (PST) Subject: -- redacted -- In-Reply-To: <6a5767b4-ef13-4ee8-8f57-3964b109ca79@googlegroups.com> References: <6a5767b4-ef13-4ee8-8f57-3964b109ca79@googlegroups.com> Message-ID: <788aca1f-0d8d-48ce-895a-9b28c9ddf9de@googlegroups.com> -- redacted -- From ginobusciarello at outlook.com Sun Feb 1 08:57:39 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 1 Feb 2015 05:57:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: <932064e4-d5b5-410a-990c-15a20def69a5@googlegroups.com> References: <932064e4-d5b5-410a-990c-15a20def69a5@googlegroups.com> Message-ID: <3008687a-0adb-4999-aa31-5184596fe573@googlegroups.com> -- redacted -- From ginobusciarello at outlook.com Sun Feb 1 08:57:56 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 1 Feb 2015 05:57:56 -0800 (PST) Subject: -- redacted -- In-Reply-To: <7757dbc7-412a-4a42-8418-801a4d5d8fb4@googlegroups.com> References: <7757dbc7-412a-4a42-8418-801a4d5d8fb4@googlegroups.com> Message-ID: <30bc2871-7824-4e80-b0a5-10662bb3eab9@googlegroups.com> -- redacted -- From steve+comp.lang.python at pearwood.info Sun Feb 1 11:20:57 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 03:20:57 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Rustom Mody wrote: > The other day I was taking a class in which I was showing > - introspection for discovering -- help, type, dir etc at the repl > - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) > > And a student asked me the diff between > dir([]) > and > [].__dir__() > > I didnt know what to say... Surely the right answer would have been "I don't know, let's check the interactive interpreter first, and the docs second." Checking the REPL first would have revealed that [].__dir__ raises AttributeError. In other words, lists don't have a __dir__ method. Checking the docs would then have revealed that __dir__ is only required when a class wishes to customise the output of dir(). Or at least, that's what I recall them saying. I'm too lazy to look it up :-) Oh very well, because it's you... https://docs.python.org/2/library/functions.html#dir Note that there are two inaccuracies in the documentation for __dir__: [quote] If the object has a method named __dir__(), this method will be called and must return the list of attributes. [end quote] The first inaccuracy is that like all (nearly all?) dunder methods, Python only looks for __dir__ on the class, not the instance itself. Hence: py> class K(object): ... def __dir__(self): ... return ["eggs", "spam"] ... py> k = K() py> from types import MethodType py> k.__dir__ = MethodType(lambda self: ["cheese", "spam"], k) py> dir(k) ['eggs', 'spam'] Well, actually that's only true for new-style classes. For old-style classic classes (Python 2 only), it will work on the instance as well. That is, in Python 2 only, if I had left out the "object" base class, dir(k) would have returned ["cheese", "spam"]. Now that I have confused your students, we shall never mention classic classes again (until next time). The second inaccuracy is that method should not return the attributes themselves, but their names. > Now surely the amount of python I dont know is significantly larger than > what I know Still it would be nice to have surface-syntax ?? dunder-magic > more systematically documented Each function or operator is unique. However, there are certain general behaviours that typically hold. Dunder methods are only looked up on the class, not the instance. (Except for classic classes, if there are any exceptions to this rule, it is probably a bug.) So the correct analog of surface syntax `obj + spam` is *not* `obj.__add__(spam)` but actually: type(obj).__add__(spam) Similar for other operators and functions. In the case of dir, the comparison should be: dir([]) versus type([]).__dir__() Except of course dir doesn't require there to be a __dir__ method. len tries to call __len__ if it exists, and if not, it tries walking the iterable counting items. bool tries calling __bool__ (Python 3) or __nonzero__ (Python 2), if it exists. If not, it tries returning len(obj) != 0. If that fails, objects are true by default. In the case of operators, operator syntax `x $ y` for some operator $ will generally do something like this: X = type(x) Y = type(y) if issubclass(Y, X): # Try the reflected version first, if it exists. dunder = getattr(Y, "__rdunder__", None) if dunder: result = dunder(y, x) if result is not NotImplemented: return result # Or the non-reflected version second. dunder = getattr(X, "__dunder__", None) if dunder: result = dunder(x, y) if result is not NotImplemented: return result else: # Like the above, except we check the non-reflected # version first, and the reflected version second. ... raise TypeError Some methods are their own reflection, e.g. __eq__. In the special case of __eq__ and __ne__, if the class defines one method but not the other, recent versions of Python will automatically call the other method and return its boolean not. Unary operators obviously have no concept of a reflected version. Comparison operators prefer to call the rich comparison methods but may fall back on the older __cmp__ dunder method (Python 2 only). -- Steven From skip.montanaro at gmail.com Sun Feb 1 11:24:03 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 10:24:03 -0600 Subject: pydoc3.5 borks on my Mac Message-ID: I build several versions of Python from Mercurial sources on a regular basis. I am starting on a new collection of programs for some database work and figured I should start using Python 3. I wanted to look something up quickly about the sqlite3 module, so I tried "pydoc sqlite3". That worked fine, but I figured it was probably using Python 2.x. In fact, it was using Python 2.6. So, I found all my pydoc-ish commands: % like pydoc /Users/skip/local/bin/pydoc /Users/skip/local/bin/pydoc3 /Users/skip/local/bin/pydoc3.2 /Users/skip/local/bin/pydoc3.3 /Users/skip/local/bin/pydoc3.4 /Users/skip/local/bin/pydoc3.5 /usr/bin/pydoc /usr/bin/pydoc2.6 /usr/bin/pydoc2.7 /usr/local/bin/pydoc3 /usr/local/bin/pydoc3.4 and tried pydoc3.5. That didn't work so well: % pydoc3.5 sqlite3 Traceback (most recent call last): File "/Users/skip/local/bin/pydoc3.5", line 5, in pydoc.cli() File "/Users/skip/local/lib/python3.5/pydoc.py", line 2591, in cli help.help(arg) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1874, in help elif request: doc(request, 'Help on %s:', output=self._output) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1612, in doc pager(render_doc(thing, title, forceload)) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1412, in pager pager(text) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1434, in return lambda text: pipepager(text, 'less') File "/Users/skip/local/lib/python3.5/pydoc.py", line 1455, in pipepager pipe.write(text) UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 600: ordinal not in range(128) I don't have PAGER or LANG set, but if I do, I get basically the same error: % LANG=en_US.utf8 PAGER=less pydoc3.5 sqlite3 Traceback (most recent call last): File "/Users/skip/local/bin/pydoc3.5", line 5, in pydoc.cli() File "/Users/skip/local/lib/python3.5/pydoc.py", line 2591, in cli help.help(arg) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1874, in help elif request: doc(request, 'Help on %s:', output=self._output) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1612, in doc pager(render_doc(thing, title, forceload)) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1412, in pager pager(text) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1428, in return lambda text: pipepager(text, os.environ['PAGER']) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1455, in pipepager pipe.write(text) UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 600: ordinal not in range(128) I get the same thing from a python3.5 prompt using "import sqlite3 ; help(sqlite3)". Not having any idea what was in position 600, I tried getting help for each element in dir(sqlite3). That failed as well, though it appeared for some strange reason to crap out on help(sqlite3.dbapi2.__name__). Any idea how I can track down this character so I can report a proper bug? Thx, Skip From steve+comp.lang.python at pearwood.info Sun Feb 1 11:31:12 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 03:31:12 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> Message-ID: <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Paul Rubin wrote: > Steven D'Aprano writes: >> Some degree of weakness in a type system is not necessarily bad. Even the >> strongest of languages usually allow a few exceptions, such as numeric >> coercions. > > Haskell doesn't have automatic coercions of any sort. You have to call > a conversion function if you want to turn an Int into an Integer. That's unusual but not unheard of. >> I've never come across a language that has pointers which insists on >> having a separate Nil pointer for ever pointer type > > Haskell's idiomatic substitute for a null pointer is a Nothing value > (like Python's None) and there's a separate one for every type. The FFI > offers actual pointers (Foreign.Ptr) and there is a separate nullPtr > for every type. Well, I live and learn. >> the compiler will allow Nil to be used for any pointer type. Anything >> else would be impractical. > > It's completely practical: polymorphism and type inference get you the > value you want with usually no effort on your part. But it's the "usually" that bites you. If I have an arbitrary pointer, and I want to check if it is safe to dereference, how do I do it? Surely I'm not expected to write something like: if type(ptr) == A: if ptr != Anil: ... if type(ptr) == B: if ptr != Bnil: ... etc. That would be insane. So how does Haskell do this? >> What if you add two empty objects? >> js> {} + {} > > OMG, javascript is worse than I thought > >> https://www.destroyallsoftware.com/talks/wat > > Can't view, needs flash. :( At the bottom of the page is a link to a .mov version. If you can't play .mov files either, contact me off list. > Try this instead (NFSW): https://www.youtube.com/watch?v=FJ7QsEytQq4 -- Steven From steve+comp.lang.python at pearwood.info Sun Feb 1 11:34:27 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 03:34:27 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> Message-ID: <54ce5593$0$12989$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > It's really only dynamically typed languages that have a single null > value of a single type. Maybe I misunderstand the original statement. Pascal is statically typed and has a single null pointer compatible with all pointer types. C has a single nil pointer compatible with all pointer types. I expect that the Modula and Oberon family of languages copied Pascal, which probably copied Algol. -- Steven From skip.montanaro at gmail.com Sun Feb 1 11:37:04 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 10:37:04 -0600 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: I finally got sort of smart, and started up a pydoc server. Looking at the sqlite3 docs through my browser, I see right off the bat, I see #-*- coding: ISO-8859-1 -*- # pysqlite2/__init__.py: the pysqlite2 package. # # Copyright (C) 2005 Gerhard H?ring so my challenge is how to tell Python my terminal's encoding? It's clearly assuming ASCII. Mine is actually normally set to UTF-8, but I can select from a number of different encodings in the Preferences. Thx, Skip From steve+comp.lang.python at pearwood.info Sun Feb 1 11:49:44 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 03:49:44 +1100 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> Message-ID: <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> Marko Rauhamaa wrote: > The natural language has a rigorous grammar plus a lexicon that includes > a number of idioms. Nobody has so far been able to codify a natural > language completely because the rigorous grammar consists of maybe > 10,000 rules. If nobody has codified the rigorous grammar, how do they know it is rigorous? Given how natural languages are constantly in flux (in both space and time), I don't even know how you could define all the rules of a grammar rigorously. By the time you finished, it would be different. Of course, some rules are more rigorous than others. No English speaker would consider a sentence like Can me cheeseburger has? as grammatical. But how about: I can has cheezburger? If grammar is defined by usage, not formal rule books (which don't exist!), then surely LOLspeak is grammatical. (It's probably closer to an argot than a full language, but even so.) -- Steven From jeanpierreda at gmail.com Sun Feb 1 11:54:42 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 1 Feb 2015 08:54:42 -0800 Subject: Python is DOOMED! Again! In-Reply-To: <54ce5593$0$12989$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce5593$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 1, 2015 at 8:34 AM, Steven D'Aprano wrote: > Devin Jeanpierre wrote: > >> It's really only dynamically typed languages that have a single null >> value of a single type. Maybe I misunderstand the original statement. > > Pascal is statically typed and has a single null pointer compatible with all > pointer types. C has a single nil pointer compatible with all pointer > types. I expect that the Modula and Oberon family of languages copied > Pascal, which probably copied Algol. No, C has a NULL macro which evaluates to something which coerces to any pointer type and will be the null value of that type. But there's one null value per type. The C standard makes no guarantees that they are compatible in any way, e.g. they can be of different sizes. On some systems, the null function pointer will have a size of N, where the null int pointer will have a size of M, where N != M -- so these are clearly not the same null value. I don't know Pascal, but I wouldn't be surprised if something similar held, as nonuniform pointer sizes were a thing once. -- Devin From steve+comp.lang.python at pearwood.info Sun Feb 1 11:55:45 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 03:55:45 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ce5a92$0$12996$c3e8da3$5496439d@news.astraweb.com> Steven D'Aprano wrote: > len tries to call __len__ if it exists, and if not, it tries walking the > iterable counting items. Hmmm, I may have mis-remembered that. Perhaps I'm thinking of Ruby. -- Steven From steve+comp.lang.python at pearwood.info Sun Feb 1 12:09:07 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 04:09:07 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce5593$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ce5db4$0$12986$c3e8da3$5496439d@news.astraweb.com> Steven D'Aprano wrote: > Devin Jeanpierre wrote: > >> It's really only dynamically typed languages that have a single null >> value of a single type. Maybe I misunderstand the original statement. > > Pascal is statically typed and has a single null pointer compatible with > all pointer types. I'm not having a good night... In Pascal, it is spelled *nil*, not null, and it is a keyword. > C has a single nil pointer compatible with all pointer > types. And in C it is null, which apparently isn't an actual value but a macro. > I expect that the Modula and Oberon family of languages copied > Pascal, which probably copied Algol. At least I was right about Modula: The constant NIL is compatible with all pointer types, and designates a pointer that does not point to any object. NIL can be assigned to any pointer type, and any pointer type can be compared to NIL. http://www.modula2.org/reference/pointertypes.php Likewise for Oberon and Oberon-2. -- Steven From __peter__ at web.de Sun Feb 1 12:20:03 2015 From: __peter__ at web.de (Peter Otten) Date: Sun, 01 Feb 2015 18:20:03 +0100 Subject: pydoc3.5 borks on my Mac References: Message-ID: Skip Montanaro wrote: > I finally got sort of smart, and started up a pydoc server. Looking at > the sqlite3 docs through my browser, I see right off the bat, I see > > #-*- coding: ISO-8859-1 -*- > # pysqlite2/__init__.py: the pysqlite2 package. > # > # Copyright (C) 2005 Gerhard H?ring > > so my challenge is how to tell Python my terminal's encoding? It's > clearly assuming ASCII. Mine is actually normally set to UTF-8, but I > can select from a number of different encodings in the Preferences. Try setting the environment variable PYTHONIOENCODING=UTF-8 From ian.g.kelly at gmail.com Sun Feb 1 12:31:45 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 1 Feb 2015 10:31:45 -0700 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54ce5a92$0$12996$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ce5a92$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 1, 2015 at 9:55 AM, Steven D'Aprano wrote: > Steven D'Aprano wrote: > >> len tries to call __len__ if it exists, and if not, it tries walking the >> iterable counting items. > > Hmmm, I may have mis-remembered that. Perhaps I'm thinking of Ruby. I think you just got it backward. iter will call __iter__ if it exists, and will try to fall back on __len__ and __getitem__ to iterate otherwise. From jeanpierreda at gmail.com Sun Feb 1 12:45:55 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 1 Feb 2015 09:45:55 -0800 Subject: Python is DOOMED! Again! In-Reply-To: <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 1, 2015 at 8:31 AM, Steven D'Aprano wrote: > Paul Rubin wrote: >> It's completely practical: polymorphism and type inference get you the >> value you want with usually no effort on your part. > > But it's the "usually" that bites you. > > If I have an arbitrary pointer, and I want to check if it is safe to > dereference, how do I do it? Surely I'm not expected to write something > like: > > if type(ptr) == A: > if ptr != Anil: ... > if type(ptr) == B: > if ptr != Bnil: ... > > etc. That would be insane. So how does Haskell do this? Haskell has different nulls in the same sense Java does: there's one keyword, whose type varies by context. Unlike Java, there is no way at all to cast different nulls to different types. Haskell has return value polymorphism and generics, so it's very easy for a function to return values of different types depending on type parameters. So this isn't even compiler hackery, it's ordinary. Also, you don't dereference in Haskell, you unpack. Python and Haskell code: if x is None: print("Not found!") else: print x case x of Nothing -> putStrLn "Not found" Some y -> putStrLn (show y) Both of these work whenever x is something that can be null and can be shown -- in Haskell, that's anything of type Maybe T, where you have access to a Show implementation for T. In Python, None is its own type/value, in Haskell there is an incompatible Nothing for each T. -- Devin From invalid at invalid.invalid Sun Feb 1 13:14:53 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 1 Feb 2015 18:14:53 +0000 (UTC) Subject: Create dictionary based of x items per key from two lists References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On 2015-02-01, Chris Angelico wrote: > On Sun, Feb 1, 2015 at 2:06 PM, Mark Lawrence wrote: >> The one-liner might not be better code, but it must be better speed wise >> precisely because it's on one line, right? :) > > Well of course it is. Python code speed is always measured in lines > per minute. That's why you should eliminate blank lines from your > code. No, you've got that backwards. You want _more_ blank lines. A blank line takes zero time to run, but it still counts as a line in your lines/second stats. A loop containing 1 line of code will execute in the same abount of time as that loop with 1 line of code and 99 blanks lines. The latter loop is running at 100 times as many lines/second as the former. That's _got_ to be better. -- Grant From skip.montanaro at gmail.com Sun Feb 1 13:21:27 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 12:21:27 -0600 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: On Sun, Feb 1, 2015 at 11:20 AM, Peter Otten <__peter__ at web.de> wrote: > Try setting the environment variable > > PYTHONIOENCODING=UTF-8 Thanks, but that didn't help. I still get the same exception. Skip From breamoreboy at yahoo.co.uk Sun Feb 1 13:49:35 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 01 Feb 2015 18:49:35 +0000 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On 01/02/2015 18:14, Grant Edwards wrote: > On 2015-02-01, Chris Angelico wrote: >> On Sun, Feb 1, 2015 at 2:06 PM, Mark Lawrence wrote: >>> The one-liner might not be better code, but it must be better speed wise >>> precisely because it's on one line, right? :) >> >> Well of course it is. Python code speed is always measured in lines >> per minute. That's why you should eliminate blank lines from your >> code. > > No, you've got that backwards. You want _more_ blank lines. A blank > line takes zero time to run, but it still counts as a line in your > lines/second stats. > > A loop containing 1 line of code will execute in the same abount of > time as that loop with 1 line of code and 99 blanks lines. > > The latter loop is running at 100 times as many lines/second as the > former. That's _got_ to be better. > At long last my quest seeking the final entry for the Zen of Python is over. I'll be delighted to let you accept the honour of raising an issue on the bug tracker to get the "this" module changed to reflect my new found wisdom. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Sun Feb 1 13:51:55 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 1 Feb 2015 11:51:55 -0700 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On Sat, Jan 31, 2015 at 7:38 PM, Jason Friedman wrote: >> I have two lists >> >> l1 = ["a","b","c","d","e","f","g","h","i","j"] >> l2 = ["aR","bR","cR"] >> >> l2 will always be smaller or equal to l1 >> >> numL1PerL2 = len(l1)/len(l2) >> >> I want to create a dictionary that has key from l1 and value from l2 based on numL1PerL2 >> >> So >> >> { >> a:aR, >> b:aR, >> c:aR, >> d:bR, >> e:bR, >> f:bR, >> g:cR, >> h:cR, >> i:cR, >> j:cR >> } > > Another possibility is: > import itertools > my_dict = {x:y for x,y in zip(list1, itertools.cycle(list2))} That results in a different mapping than the one specified by the OP, though. An alternate itertools approach could be: dict(zip_longest(l1, chain.from_iterable( map(partial(repeat, times=len(l1) // len(l2)), l2)), fillvalue=l2[-1])) although this requires the inputs to be sequences, not arbitrary iterables. From none at mailinator.com Sun Feb 1 13:52:09 2015 From: none at mailinator.com (mm0fmf) Date: Sun, 01 Feb 2015 18:52:09 +0000 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On 01/02/2015 18:14, Grant Edwards wrote: > No, you've got that backwards. You want_more_ blank lines. A blank > line takes zero time to run, but it still counts as a line in your > lines/second stats. You want more blanks lines to satisfy bean counting managers who want you to report "number of lines of code written per week". More blank lines equals happier managers! ;-) From marko at pacujo.net Sun Feb 1 13:57:22 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 01 Feb 2015 20:57:22 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87twz5e9yl.fsf@elektro.pacujo.net> Steven D'Aprano : > Marko Rauhamaa wrote: > >> The natural language has a rigorous grammar plus a lexicon that includes >> a number of idioms. Nobody has so far been able to codify a natural >> language completely because the rigorous grammar consists of maybe >> 10,000 rules. > > If nobody has codified the rigorous grammar, how do they know it is > rigorous? Even a five-year-old will immediately spot grammar mistakes (and let you know about them!). > Given how natural languages are constantly in flux (in both space and time), > I don't even know how you could define all the rules of a grammar > rigorously. By the time you finished, it would be different. Every brain has a slightly different variant, and that is evolving, too. However, you can spot a non-native speaker (or a speaker outside your community) in a fraction of a second after they open their mouth. > If grammar is defined by usage, not formal rule books (which don't > exist!), then surely LOLspeak is grammatical. (It's probably closer to > an argot than a full language, but even so.) Whatever it is, your brain will recognize its native language. You never make a grammar mistake that you don't spot yourself right away. Marko From emile at fenx.com Sun Feb 1 14:07:28 2015 From: emile at fenx.com (Emile van Sebille) Date: Sun, 01 Feb 2015 11:07:28 -0800 Subject: CSV and number formats In-Reply-To: References: Message-ID: On 1/31/2015 10:45 PM, Frank Millman wrote: > If the opening balance is positive, it appears as '+00000021.45' > > If it is negative, it appears as '+00000-21.45' My advise is to get cash in payment. :) Emile From marko at pacujo.net Sun Feb 1 14:12:49 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 01 Feb 2015 21:12:49 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> Message-ID: <87oapde98u.fsf@elektro.pacujo.net> Christian Gollwitzer : > Am 01.02.15 um 08:58 schrieb Marko Rauhamaa: >> Qt gave up on C++ when it comes to callbacks ("signals") and went for >> an apocryphal metacompiler. > > Yes, but only because C++ compilers were not good enough when QT came > out, and later is was too late to change it to a templated system. > Lookup libsigc++ http://libsigc.sourceforge.net/ which does the same > using standard C++ and http://qt-project.org/doc/qt-4.8/templates.html > for a reasoning of QT. > > In C++11 (supported by MSVC, g++, clang) there re also lambda expressions So please implement this small piece of Python code in C++ so we can compare the idioms: import sys, time class MyLogger: def __init__(self, write): self.write = sys.stderr.write def register_writer(self, write): self.write = write def log(self, text): self.write("{} {}\n".format(time.time(), text)) class MyApp: def __init__(self, logger): self.logf = open("thelog.log", "a") logger.register_writer(self.log_write) def log_write(self, text): self.logf.write(text) self.logf.flush() Marko From __peter__ at web.de Sun Feb 1 14:35:39 2015 From: __peter__ at web.de (Peter Otten) Date: Sun, 01 Feb 2015 20:35:39 +0100 Subject: pydoc3.5 borks on my Mac References: Message-ID: Skip Montanaro wrote: > On Sun, Feb 1, 2015 at 11:20 AM, Peter Otten <__peter__ at web.de> wrote: >> Try setting the environment variable >> >> PYTHONIOENCODING=UTF-8 > > Thanks, but that didn't help. I still get the same exception. The pager is invoked by os.popen(), and after some digging I find that it uses a io.TestIOWrapper() to write the help text. This in turn uses locale.getpreferredencoding(False), i. e. you were right to set LANG and PYTHONIOENCODING is not relevant. I'm able to provoke what I think causes your problem on linux with Python 3.4: $ LANG=en_US.utf-8 python3 -c 'import os; os.popen("cat", "w").write("\xe4\n")' ? $ LANG=en_US.ascii python3 -c 'import os; os.popen("cat", "w").write("\xe4\n")' Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 0: ordinal not in range(128) Perhaps LANG=en_US.utf-8 is not understood by your system (uppercase UTF-8? I really don't know the Mac). What does $ LANG=en_US.UTF-8 python3 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 print? From ian.g.kelly at gmail.com Sun Feb 1 14:36:14 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 1 Feb 2015 12:36:14 -0700 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On Sun, Feb 1, 2015 at 11:49 AM, Mark Lawrence wrote: > On 01/02/2015 18:14, Grant Edwards wrote: >> A loop containing 1 line of code will execute in the same abount of >> time as that loop with 1 line of code and 99 blanks lines. >> >> The latter loop is running at 100 times as many lines/second as the >> former. That's _got_ to be better. >> > > At long last my quest seeking the final entry for the Zen of Python is over. > I'll be delighted to let you accept the honour of raising an issue on the > bug tracker to get the "this" module changed to reflect my new found wisdom. And while you're at it, please also work on changing the name of the module to "self" -- "this" is for Java. From skip.montanaro at gmail.com Sun Feb 1 15:17:46 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 14:17:46 -0600 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: On Sun, Feb 1, 2015 at 1:35 PM, Peter Otten <__peter__ at web.de> wrote: > $ LANG=en_US.UTF-8 python3 -c 'import locale; print(locale.getpreferredencoding(False))' Aha! hgpython% LANG=en_US.UTF-8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 hgpython% LANG=en_US.utf8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' US-ASCII Thank you. I always thought these Unicode encodings were supposed to be case-insensitive. Skip From breamoreboy at yahoo.co.uk Sun Feb 1 15:28:42 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 01 Feb 2015 20:28:42 +0000 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On 01/02/2015 19:36, Ian Kelly wrote: > On Sun, Feb 1, 2015 at 11:49 AM, Mark Lawrence wrote: >> On 01/02/2015 18:14, Grant Edwards wrote: >>> A loop containing 1 line of code will execute in the same abount of >>> time as that loop with 1 line of code and 99 blanks lines. >>> >>> The latter loop is running at 100 times as many lines/second as the >>> former. That's _got_ to be better. >>> >> >> At long last my quest seeking the final entry for the Zen of Python is over. >> I'll be delighted to let you accept the honour of raising an issue on the >> bug tracker to get the "this" module changed to reflect my new found wisdom. > > And while you're at it, please also work on changing the name of the > module to "self" -- "this" is for Java. > I think that'll need a PEP :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Sun Feb 1 15:31:55 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 07:31:55 +1100 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: On Mon, Feb 2, 2015 at 7:17 AM, Skip Montanaro wrote: > Thank you. I always thought these Unicode encodings were supposed to > be case-insensitive. I'd have thought so, too. Try the other variations: include the hyphen but don't capitalize, and the other way around. On my system, all four work equally: rosuav at sikorsky:~$ echo $LANG en_AU.UTF-8 rosuav at sikorsky:~$ LANG=en_AU.utf8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 rosuav at sikorsky:~$ LANG=en_AU.utf-8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 rosuav at sikorsky:~$ LANG=en_AU.UTF8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 rosuav at sikorsky:~$ LANG=en_AU.UTF-8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' UTF-8 However, and very interestingly, en_US does not: rosuav at sikorsky:~$ LANG=en_US.UTF-8 python3.5 -c 'import locale; print(locale.getpreferredencoding(False))' ANSI_X3.4-1968 Which is probably because I don't have US locales installed. Maybe there's something about exactly what's installed?? ChrisA From rosuav at gmail.com Sun Feb 1 15:33:19 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 07:33:19 +1100 Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On Mon, Feb 2, 2015 at 6:36 AM, Ian Kelly wrote: >> At long last my quest seeking the final entry for the Zen of Python is over. >> I'll be delighted to let you accept the honour of raising an issue on the >> bug tracker to get the "this" module changed to reflect my new found wisdom. > > And while you're at it, please also work on changing the name of the > module to "self" -- "this" is for Java. I thought it was meant to be like English - "Here, import this". ChrisA From rosuav at gmail.com Sun Feb 1 15:45:55 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 07:45:55 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 1, 2015 at 4:36 PM, Rustom Mody wrote: > The other day I was taking a class in which I was showing > - introspection for discovering -- help, type, dir etc at the repl > - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) > > And a student asked me the diff between > dir([]) > and > [].__dir__() > > I didnt know what to say... Simple answer: You write dunder methods and the interpreter calls them. You don't call them yourself. I can't currently think of any situation where it's appropriate to call a dunder method manually (cue the swamping of such situations on the list); you just call dir() or the + operator or whatever it be. ChrisA From emile at fenx.com Sun Feb 1 15:51:58 2015 From: emile at fenx.com (Emile van Sebille) Date: Sun, 01 Feb 2015 12:51:58 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/1/2015 12:45 PM, Chris Angelico wrote: > Simple answer: You write dunder methods and the interpreter calls > them. You don't call them yourself. > > I can't currently think of any situation where it's appropriate to > call a dunder method manually (cue the swamping of such situations on > the list); you just call dir() or the + operator or whatever it be. calling the parent dunder method from within the overiding new method comes to mind. Emile From skip.montanaro at gmail.com Sun Feb 1 16:01:14 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 1 Feb 2015 15:01:14 -0600 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: On Sun, Feb 1, 2015 at 2:31 PM, Chris Angelico wrote: > Try the other variations: include the hyphen > but don't capitalize, and the other way around. On my system, all four > work equally: Yes, on my system, case doesn't matter, but the hyphen does. I just tried it out on one of my Linux systems at work. It's an aging openSuSE 12.2 system, so all I could install was 3.2.3, but it worked exactly like your system. Any capitalization, with or without the hyphen, and the pydoc3.2 command worked. With LANG=C or with LANG unset, I get the UnicodeEncodeError traceback. FYI, I did open a bug report, and tried to transfer the useful bits from this thread to it: http://bugs.python.org/issue23374 Thanks to Peter and Chris. I'm off and running. Well, until I stumble over the next tree root or crack in the sidewalk... Skip From no.email at nospam.invalid Sun Feb 1 17:02:04 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 14:02:04 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce5593$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87wq412sv7.fsf@jester.gateway.sonic.net> Steven D'Aprano writes: > C has a single nil pointer compatible with all pointer types. C++11 has a separate type just for the null pointer, which can be automatically coerced to other pointer types. I'm not sure but I think that means it is couthing up slightly. http://en.cppreference.com/w/cpp/types/nullptr_t http://en.cppreference.com/w/cpp/language/nullptr From cs at zip.com.au Sun Feb 1 17:09:43 2015 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 2 Feb 2015 09:09:43 +1100 Subject: pydoc3.5 borks on my Mac In-Reply-To: References: Message-ID: <20150201220943.GA52147@cskk.homeip.net> On 01Feb2015 14:17, Skip Montanaro wrote: >On Sun, Feb 1, 2015 at 1:35 PM, Peter Otten <__peter__ at web.de> wrote: >> $ LANG=en_US.UTF-8 python3 -c 'import locale; print(locale.getpreferredencoding(False))' > >Aha! > >hgpython% LANG=en_US.UTF-8 python3.5 -c 'import locale; >print(locale.getpreferredencoding(False))' >UTF-8 >hgpython% LANG=en_US.utf8 python3.5 -c 'import locale; >print(locale.getpreferredencoding(False))' >US-ASCII > >Thank you. I always thought these Unicode encodings were supposed to >be case-insensitive. The latter lacks a dash. See if it's that. Cheers, Cameron Simpson A feature is a bug with seniority. From no.email at nospam.invalid Sun Feb 1 17:27:19 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 14:27:19 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> Message-ID: <87siep2rp4.fsf@jester.gateway.sonic.net> Devin Jeanpierre writes: > That said, Haskell (and the rest) do have a sort of type coercion, of > literals at compile time (e.g. 3 can be an Integer or a Double > depending on how you use it.) That's polymorphism, not coercion. The compiler figures out at compile time what type of 3 you actually mean: there is never an automatic runtime conversion. sqrt(3) works because sqrt expects a floating argument so the compiler deduces that the 3 that you wrote denotes a float. sqrt(3+length(xs)) has to fail because length returns an int, so 3+length(xs) is an int, and you can't pass an int to sqrt. > BTW it's weird that in this thread, and in the programmer community at > large, int->string is considered worse than int->float Hehe, though int->string leads to plenty of weird bugs. >> Haskell's idiomatic substitute for a null pointer is a Nothing value > For that matter, how is this (first part) different from, say, Java? In Java, functions expecting to receve sensible values can get null by surprise. In Haskell, if a term can have a Nothing value, that has to be reflected in its type. Haskell's bug-magnet counterpart to Java's null values is Bottom, an artifact of lazy evaluation. E.g. you can write x = 3 / 0 someplace in your program, and the program will accept this and run merrily until you try to actually print something that depends on x, at which point it crashes. From jeanpierreda at gmail.com Sun Feb 1 17:52:22 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 1 Feb 2015 14:52:22 -0800 Subject: Python is DOOMED! Again! In-Reply-To: <87siep2rp4.fsf@jester.gateway.sonic.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <87siep2rp4.fsf@jester.gateway.sonic.net> Message-ID: On Sun, Feb 1, 2015 at 2:27 PM, Paul Rubin wrote: > Devin Jeanpierre writes: >> That said, Haskell (and the rest) do have a sort of type coercion, of >> literals at compile time (e.g. 3 can be an Integer or a Double >> depending on how you use it.) > > That's polymorphism, not coercion. OK, yes, that fits better into how Haskell works. After all, that's how "Nothing" works. If 3 is just a (magic) constructor, then it's no different. > The compiler figures out at compile > time what type of 3 you actually mean: there is never an automatic > runtime conversion. sqrt(3) works because sqrt expects a floating > argument so the compiler deduces that the 3 that you wrote denotes a > float. sqrt(3+length(xs)) has to fail because length returns an int, so > 3+length(xs) is an int, and you can't pass an int to sqrt. > >> BTW it's weird that in this thread, and in the programmer community at >> large, int->string is considered worse than int->float > > Hehe, though int->string leads to plenty of weird bugs. > >>> Haskell's idiomatic substitute for a null pointer is a Nothing value >> For that matter, how is this (first part) different from, say, Java? > > In Java, functions expecting to receve sensible values can get null by > surprise. In Haskell, if a term can have a Nothing value, that has to > be reflected in its type. Haskell's bug-magnet counterpart to Java's > null values is Bottom, an artifact of lazy evaluation. E.g. you can > write > x = 3 / 0 > someplace in your program, and the program will accept this and run > merrily until you try to actually print something that depends on x, > at which point it crashes. This isn't a difference in whether there are multiple nulls, though. I answered my own question later, by accident: Java nulls are castable to each other if you do it explicitly (routing through Object -- e.g. (Something)((Object) ((SomeOtherThing) null)))). So in that sense, there is only one null, just with some arbitrary compiler distinctions you can break through if you try hard enough. -- Devin From torriem at gmail.com Sun Feb 1 19:50:56 2015 From: torriem at gmail.com (Michael Torrie) Date: Sun, 01 Feb 2015 17:50:56 -0700 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <87oapde98u.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> Message-ID: <54CEC9F0.7080004@gmail.com> On 02/01/2015 12:12 PM, Marko Rauhamaa wrote: > Christian Gollwitzer : > >> Am 01.02.15 um 08:58 schrieb Marko Rauhamaa: >>> Qt gave up on C++ when it comes to callbacks ("signals") and went for >>> an apocryphal metacompiler. >> >> Yes, but only because C++ compilers were not good enough when QT came >> out, and later is was too late to change it to a templated system. >> Lookup libsigc++ http://libsigc.sourceforge.net/ which does the same >> using standard C++ and http://qt-project.org/doc/qt-4.8/templates.html >> for a reasoning of QT. >> >> In C++11 (supported by MSVC, g++, clang) there re also lambda expressions > > So please implement this small piece of Python code in C++ so we can > compare the idioms: Honestly with the C++ standard library implementing std::function and std::bind macros, idiomatically it would look very much similar to the Python code you showed. From torriem at gmail.com Sun Feb 1 20:02:19 2015 From: torriem at gmail.com (Michael Torrie) Date: Sun, 01 Feb 2015 18:02:19 -0700 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <87oapde98u.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> Message-ID: <54CECC9B.4050704@gmail.com> On 02/01/2015 12:12 PM, Marko Rauhamaa wrote: > So please implement this small piece of Python code in C++ so we can > compare the idioms: So I though I might just for kicks code up a C++ version. In doing so, I realized that idomatically, this particular example would not really use callbacks in real life, but rather stream objects. Would look nearly the same but be more idiomatic C++. However, C++ function pointers are actually pretty slick in C++11. See the example near the bottom of the page: http://en.cppreference.com/w/cpp/utility/functional/function Thus if we were to shoehorn your example into C++, the result would be idiomatically very similar to what you have in your Python code. From torriem at gmail.com Sun Feb 1 20:02:46 2015 From: torriem at gmail.com (Michael Torrie) Date: Sun, 01 Feb 2015 18:02:46 -0700 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <54CEC9F0.7080004@gmail.com> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> <54CEC9F0.7080004@gmail.com> Message-ID: <54CECCB6.2040003@gmail.com> On 02/01/2015 05:50 PM, Michael Torrie wrote: > Honestly with the C++ standard library implementing std::function and > std::bind macros, idiomatically it would look very much similar to the > Python code you showed. Make that templates, not macros. From steve+comp.lang.python at pearwood.info Sun Feb 1 20:59:38 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 12:59:38 +1100 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> Message-ID: <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Marko Rauhamaa wrote: > Steven D'Aprano : > >> Marko Rauhamaa wrote: >> >>> The natural language has a rigorous grammar plus a lexicon that includes >>> a number of idioms. Nobody has so far been able to codify a natural >>> language completely because the rigorous grammar consists of maybe >>> 10,000 rules. >> >> If nobody has codified the rigorous grammar, how do they know it is >> rigorous? > > Even a five-year-old will immediately spot grammar mistakes (and let you > know about them!). That doesn't make it rigorous. That just means that there are rules, and five-years olds know some of the rules. There may be ambiguities in the rules. There could be sentences where no rule applies, or multiple contradictory rules. Children go through a period of language acquisition where they over-generalise rules, saying things like "I eated the biscuit". Amusingly, there are at least some reports of children getting angry when their parents copy those over-generalisations, saying words to the effect of "No Daddy, I say eated, you don't!" This suggests that they know that the rule is wrong even when they say it, yet they continue to say it. This in turn suggests that there are multiple parts of the brain involved in grammar, and they aren't always in sync. Adults also commit hyper-correction. Anyone who has said a sentence like "The dog barked at my daughter and I" is being ungrammatical. Cross out the second party, and you are left with: "The dog barked at I" which is clearly wrong. And there are underspecified rules too. What is the plural of octopus? No fair looking it up in the dictionary. [...] > Whatever it is, your brain will recognize its native language. You never > make a grammar mistake that you don't spot yourself right away. Of course people make grammar mistakes that they don't spot. If you're interested in this topic, I can strongly recommend Steven Pinker's book "Words And Rules". -- Steven From steve+comp.lang.python at pearwood.info Sun Feb 1 21:03:14 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 13:03:14 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <87siep2rp4.fsf@jester.gateway.sonic.net> Message-ID: <54cedae2$0$12977$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > So in that sense, there is only one null, just with some arbitrary > compiler distinctions you can break through if you try hard enough. Woo hoo! I was right! *Dances the Dance of Victory!* -- Steven From rosuav at gmail.com Sun Feb 1 21:04:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 13:04:58 +1100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano wrote: > And there are underspecified rules too. What is the plural of octopus? No > fair looking it up in the dictionary. Standard and well-known piece of trivia, and there are several options. "Octopodes" is one of the most rigorously formal, but "octopuses" is perfectly acceptable. "Octopi" is technically incorrect, as the -us ending does not derive from the Latin. ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 1 21:05:44 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 13:05:44 +1100 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cedb79$0$12981$c3e8da3$5496439d@news.astraweb.com> Steven D'Aprano wrote: > Of course people make grammar mistakes that they don't spot. Ironically, this is one of them. It should of course be "grammatical mistakes". Seriously, I didn't do that on purpose. I only noticed the error on reading it back after sending. -- Steven From auxlang at mrabarnett.plus.com Sun Feb 1 21:35:21 2015 From: auxlang at mrabarnett.plus.com (Matthew Barnett) Date: Mon, 02 Feb 2015 02:35:21 +0000 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54CEE269.7070608@mrabarnett.plus.com> On 2015-02-02 02:04, Chris Angelico wrote: > On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano > wrote: >> And there are underspecified rules too. What is the plural of >> octopus? No fair looking it up in the dictionary. > > Standard and well-known piece of trivia, and there are several > options. "Octopodes" is one of the most rigorously formal, but > "octopuses" is perfectly acceptable. "Octopi" is technically > incorrect, as the -us ending does not derive from the Latin. > And the plural of "virus" is "viruses", not "viri" (that's the plural of "vir") or "virii" (that would be the plural of "virius", if it existed). From rosuav at gmail.com Sun Feb 1 21:45:20 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 13:45:20 +1100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <54CEE269.7070608@mrabarnett.plus.com> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <54CEE269.7070608@mrabarnett.plus.com> Message-ID: On Mon, Feb 2, 2015 at 1:35 PM, Matthew Barnett wrote: > And the plural of "virus" is "viruses", not "viri" (that's the plural of > "vir") or "virii" (that would be the plural of "virius", if it existed). Yes indeed."Virii" and "octopi" are as wrong as "hice" for "houses" (paralleling mice). ChrisA From rustompmody at gmail.com Sun Feb 1 22:52:39 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 1 Feb 2015 19:52:39 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sunday, February 1, 2015 at 9:51:11 PM UTC+5:30, Steven D'Aprano wrote: > Rustom Mody wrote: > > > The other day I was taking a class in which I was showing > > - introspection for discovering -- help, type, dir etc at the repl > > - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) > > > > And a student asked me the diff between > > dir([]) > > and > > [].__dir__() > > > > I didnt know what to say... > > Surely the right answer would have been "I don't know, let's check the > interactive interpreter first, and the docs second." > > Checking the REPL first would have revealed that [].__dir__ raises > AttributeError. In other words, lists don't have a __dir__ method. ?? $ python3 Python 3.4.2 (default, Oct 8 2014, 13:08:17) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> [].__dir__ >>> dir([]) ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] >>> [].__dir__() ['__rmul__', '__str__', '__gt__', 'remove', '__class__', '__getitem__', '__format__', '__ne__', '__sizeof__', '__contains__', '__reduce__', '__add__', 'sort', 'count', 'extend', '__mul__', '__imul__', '__reduce_ex__', '__setitem__', '__doc__', '__ge__', 'copy', '__init__', '__iadd__', '__hash__', '__delitem__', 'insert', '__iter__', '__repr__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', 'index', '__lt__', 'clear', '__subclasshook__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'] >>> What you are describing seems to be true for python2 though From rustompmody at gmail.com Sun Feb 1 23:04:41 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 1 Feb 2015 20:04:41 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: <074ea090-708a-443a-b5f8-686c2dce8078@googlegroups.com> On Monday, February 2, 2015 at 9:22:51 AM UTC+5:30, Rustom Mody wrote: > On Sunday, February 1, 2015 at 9:51:11 PM UTC+5:30, Steven D'Aprano wrote: > > Rustom Mody wrote: > > > > > The other day I was taking a class in which I was showing > > > - introspection for discovering -- help, type, dir etc at the repl > > > - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) > > > > > > And a student asked me the diff between > > > dir([]) > > > and > > > [].__dir__() > > > > > > I didnt know what to say... > > > > Surely the right answer would have been "I don't know, let's check the > > interactive interpreter first, and the docs second." > > > > Checking the REPL first would have revealed that [].__dir__ raises > > AttributeError. In other words, lists don't have a __dir__ method. > > ?? > > $ python3 > Python 3.4.2 (default, Oct 8 2014, 13:08:17) > [GCC 4.9.1] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> [].__dir__ > > >>> dir([]) > ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] > >>> [].__dir__() > ['__rmul__', '__str__', '__gt__', 'remove', '__class__', '__getitem__', '__format__', '__ne__', '__sizeof__', '__contains__', '__reduce__', '__add__', 'sort', 'count', 'extend', '__mul__', '__imul__', '__reduce_ex__', '__setitem__', '__doc__', '__ge__', 'copy', '__init__', '__iadd__', '__hash__', '__delitem__', 'insert', '__iter__', '__repr__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', 'index', '__lt__', 'clear', '__subclasshook__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'] > >>> > > What you are describing seems to be true for python2 though And since they *looked* different I believed they were different. Evidently not? >>> s1= set([].__dir__()) >>> s2=set(dir([])) >>> s1 {'__rmul__', '__str__', '__class__', '__ne__', '__repr__', '__format__', '__sizeof__', '__contains__', '__add__', 'sort', 'count', 'extend', 'remove', '__mul__', '__reduce__', '__imul__', '__reduce_ex__', '__setitem__', 'insert', '__doc__', '__ge__', 'index', 'copy', '__subclasshook__', '__getitem__', '__init__', '__iadd__', '__hash__', '__delitem__', '__iter__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', '__lt__', 'clear', '__gt__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'} >>> s2 {'__rmul__', '__str__', '__class__', '__ne__', '__repr__', '__format__', '__sizeof__', '__contains__', '__add__', 'sort', 'count', 'extend', '__mul__', 'remove', '__imul__', '__reduce__', '__reduce_ex__', '__setitem__', 'insert', '__doc__', '__ge__', '__subclasshook__', 'copy', 'index', '__getitem__', '__iadd__', '__init__', '__hash__', '__delitem__', '__iter__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', '__lt__', 'clear', '__gt__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'} >>> len(s1) 45 >>> len(s2) 45 >>> s1 - s2 set() >>> s2 - s1 set() From roy at panix.com Sun Feb 1 23:12:38 2015 From: roy at panix.com (Roy Smith) Date: Sun, 01 Feb 2015 23:12:38 -0500 Subject: [OT] fortran lib which provide python like data type References: <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <54CEE269.7070608@mrabarnett.plus.com> Message-ID: In article , Chris Angelico wrote: > On Mon, Feb 2, 2015 at 1:35 PM, Matthew Barnett > wrote: > > And the plural of "virus" is "viruses", not "viri" (that's the plural of > > "vir") or "virii" (that would be the plural of "virius", if it existed). > > Yes indeed."Virii" and "octopi" are as wrong as "hice" for "houses" > (paralleling mice). And, of course, I die, but we dice. From roy at panix.com Sun Feb 1 23:14:37 2015 From: roy at panix.com (Roy Smith) Date: Sun, 01 Feb 2015 23:14:37 -0500 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: In article <54ceda0b$0$12977$c3e8da3$5496439d at news.astraweb.com>, Steven D'Aprano wrote: > What is the plural of octopus? It's a trick question. Octopus is already plural. Monopus is singular. From rustompmody at gmail.com Sun Feb 1 23:22:44 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 1 Feb 2015 20:22:44 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <074ea090-708a-443a-b5f8-686c2dce8078@googlegroups.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <074ea090-708a-443a-b5f8-686c2dce8078@googlegroups.com> Message-ID: On Monday, February 2, 2015 at 9:34:53 AM UTC+5:30, Rustom Mody wrote: > On Monday, February 2, 2015 at 9:22:51 AM UTC+5:30, Rustom Mody wrote: > > On Sunday, February 1, 2015 at 9:51:11 PM UTC+5:30, Steven D'Aprano wrote: > > > Rustom Mody wrote: > > > > > > > The other day I was taking a class in which I was showing > > > > - introspection for discovering -- help, type, dir etc at the repl > > > > - mapping of surface syntax to internals eg. a + b ?? a.__add__(b) > > > > > > > > And a student asked me the diff between > > > > dir([]) > > > > and > > > > [].__dir__() > > > > > > > > I didnt know what to say... > > > > > > Surely the right answer would have been "I don't know, let's check the > > > interactive interpreter first, and the docs second." > > > > > > Checking the REPL first would have revealed that [].__dir__ raises > > > AttributeError. In other words, lists don't have a __dir__ method. > > > > ?? > > > > $ python3 > > Python 3.4.2 (default, Oct 8 2014, 13:08:17) > > [GCC 4.9.1] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > >>> [].__dir__ > > > > >>> dir([]) > > ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] > > >>> [].__dir__() > > ['__rmul__', '__str__', '__gt__', 'remove', '__class__', '__getitem__', '__format__', '__ne__', '__sizeof__', '__contains__', '__reduce__', '__add__', 'sort', 'count', 'extend', '__mul__', '__imul__', '__reduce_ex__', '__setitem__', '__doc__', '__ge__', 'copy', '__init__', '__iadd__', '__hash__', '__delitem__', 'insert', '__iter__', '__repr__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', 'index', '__lt__', 'clear', '__subclasshook__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'] > > >>> > > > > What you are describing seems to be true for python2 though > > And since they *looked* different I believed they were different. > > Evidently not? > > >>> s1= set([].__dir__()) > >>> s2=set(dir([])) > >>> s1 > {'__rmul__', '__str__', '__class__', '__ne__', '__repr__', '__format__', '__sizeof__', '__contains__', '__add__', 'sort', 'count', 'extend', 'remove', '__mul__', '__reduce__', '__imul__', '__reduce_ex__', '__setitem__', 'insert', '__doc__', '__ge__', 'index', 'copy', '__subclasshook__', '__getitem__', '__init__', '__iadd__', '__hash__', '__delitem__', '__iter__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', '__lt__', 'clear', '__gt__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'} > >>> s2 > {'__rmul__', '__str__', '__class__', '__ne__', '__repr__', '__format__', '__sizeof__', '__contains__', '__add__', 'sort', 'count', 'extend', '__mul__', 'remove', '__imul__', '__reduce__', '__reduce_ex__', '__setitem__', 'insert', '__doc__', '__ge__', '__subclasshook__', 'copy', 'index', '__getitem__', '__iadd__', '__init__', '__hash__', '__delitem__', '__iter__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', '__lt__', 'clear', '__gt__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__'} > >>> len(s1) > 45 > >>> len(s2) > 45 > >>> s1 - s2 > set() > >>> s2 - s1 > set() Well I continue to be fooled >>> d1 = {k:getattr([],k) for k in [].__dir__()} >>> d2 = {k:getattr([],k) for k in dir([])} >>> d1 == d2 False >>> len(d1) 45 >>> len(d2) 45 >>> d1.keys() dict_keys(['__rmul__', '__str__', '__gt__', '__mul__', '__class__', '__ne__', '__format__', '__sizeof__', '__contains__', '__imul__', '__add__', 'sort', 'count', 'extend', 'remove', '__init__', 'insert', '__setitem__', 'index', '__subclasshook__', 'copy', '__getitem__', '__iadd__', '__hash__', '__delitem__', '__reduce_ex__', '__iter__', '__repr__', '__le__', '__setattr__', 'reverse', '__new__', '__eq__', '__len__', '__doc__', '__lt__', 'clear', '__ge__', 'append', '__dir__', '__reversed__', '__getattribute__', 'pop', '__delattr__', '__reduce__']) >>> d1.keys() == d2.keys() True From rosuav at gmail.com Sun Feb 1 23:47:36 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 15:47:36 +1100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 3:14 PM, Roy Smith wrote: > In article <54ceda0b$0$12977$c3e8da3$5496439d at news.astraweb.com>, > Steven D'Aprano wrote: > >> What is the plural of octopus? > > It's a trick question. Octopus is already plural. Monopus is singular. People is already plural, too, but you can talk about all the peoples of the world. Also, I can use "people" as the subject and "is" as the verb, just to completely destroy any chance of a simple grammar parser being able to cope with English. ChrisA From ben+python at benfinney.id.au Sun Feb 1 23:53:13 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 02 Feb 2015 15:53:13 +1100 Subject: [RELEASE] =?utf-8?Q?=E2=80=98python-daemon=E2=80=99?= version 2.0.5 released Message-ID: <85wq40hq2u.fsf@benfinney.id.au> Howdy all, I am pleased to announce the release of version 2.0.5 of the ?python-daemon? library. The current release is always available at . The project's forums and VCS are hosted at Alioth . Significant changes since the previous version ============================================== This is a bug fix release, addressing these bugs: * Specify the text encoding when opening the changelog file. This addresses some reported failures of building the package from source. Thanks to those who reported problems. * Refine compatibility of exceptions for file operations. Different Python versions have drifted the hierarchy location of exceptions resulting from file operations. This release tests more conditions, and captures the correct exceptions more robustly. What is the ?python-daemon? library? ==================================== ?python-daemon? is a Python library to implement a well-behaved Unix daemon process. -- \ ?Institutions will try to preserve the problem to which they | `\ are the solution.? ?Clay Shirky, 2012 | _o__) | Ben Finney From greg.ewing at canterbury.ac.nz Sun Feb 1 23:55:21 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 02 Feb 2015 17:55:21 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > [quote] > If the object has a method named __dir__(), this method will > be called and must return the list of attributes. > [end quote] > > The first inaccuracy is that like all (nearly all?) dunder methods, Python > only looks for __dir__ on the class, not the instance itself. It says "method", not "attribute", so technically it's correct. The methods of an object are defined by what's in its class. -- Greg From greg.ewing at canterbury.ac.nz Mon Feb 2 00:19:37 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 02 Feb 2015 18:19:37 +1300 Subject: Python is DOOMED! Again! In-Reply-To: <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > If I have an arbitrary pointer, and I want to check if it is safe to > dereference, how do I do it? Surely I'm not expected to write something > like: > > if type(ptr) == A: > if ptr != Anil: ... > if type(ptr) == B: > if ptr != Bnil: ... > > etc. That would be insane. So how does Haskell do this? In Haskell you would just go ahead and compare ptr with Nothing (or more likely pattern-match it against Nothing). Haskell knows the type of the thing you're comparing to, and uses type inference to select the right type of Nothing to use. BTW, technically, Nothing isn't really a null pointer, it's a member of an algebraic type defined in the standard library: data Maybe a = Just a | Nothing So conceptually, there is a different Nothing for each possible type 'a' in Maybe a. But since the Nothing constructor doesn't have any arguments, the implementation could represent them all by the same value if it wanted. -- Greg From vito.detullio at gmail.com Mon Feb 2 00:26:55 2015 From: vito.detullio at gmail.com (Vito De Tullio) Date: Mon, 02 Feb 2015 06:26:55 +0100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > Checking the REPL first would have revealed that [].__dir__ raises > AttributeError. In other words, lists don't have a __dir__ method. ? Python 3.4.2 (default, Nov 29 2014, 00:45:45) [GCC 4.8.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> [].__dir__() ['sort', '__contains__', '__init__', '__ge__', 'count', '__class__', '__format__', '__mul__', 'index', '__rmul__', '__hash__', '__iter__', 'clear', '__subclasshook__', '__getitem__', 'reverse', 'append', '__ne__', 'pop', '__reduce__', '__add__', 'extend', '__gt__', '__sizeof__', '__setattr__', '__imul__', '__dir__', '__le__', 'insert', '__repr__', '__str__', '__getattribute__', '__len__', '__lt__', 'remove', '__new__', '__reduce_ex__', 'copy', '__reversed__', '__delattr__', '__eq__', '__setitem__', '__iadd__', '__doc__', '__delitem__'] >>> -- By ZeD From rosuav at gmail.com Mon Feb 2 00:38:36 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 16:38:36 +1100 Subject: Python is DOOMED! Again! In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 4:19 PM, Gregory Ewing wrote: > In Haskell you would just go ahead and compare ptr > with Nothing (or more likely pattern-match it against > Nothing). So since you can set something to Nothing regardless of type, and compare it against Nothing regardless of type, it doesn't really much matter that there are different types of Nothing. Right? ChrisA From greg.ewing at canterbury.ac.nz Mon Feb 2 00:46:13 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 02 Feb 2015 18:46:13 +1300 Subject: Python is DOOMED! Again! In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <87siep2rp4.fsf@jester.gateway.sonic.net> Message-ID: Devin Jeanpierre wrote: > I answered my own question later, by accident: Java nulls are castable > to each other if you do it explicitly (routing through Object -- e.g. > (Something)((Object) ((SomeOtherThing) null)))). > > So in that sense, there is only one null, just with some arbitrary > compiler distinctions you can break through if you try hard enough. You can't conclude that they all have the same runtime representation, though. The compiler could be generating code to convert them in response to the cast. -- Greg From no.email at nospam.invalid Mon Feb 2 01:07:44 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 22:07:44 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87lhkg3ky7.fsf@jester.gateway.sonic.net> Chris Angelico writes: > So since you can set something to Nothing regardless of type, and > compare it against Nothing regardless of type, it doesn't really much > matter that there are different types of Nothing. Right? No that's not how type inference works. If you have x = Nothing and pass it to a function that takes a Maybe Int, type inference means the compiler figures out that x must have type Maybe Int. If you then also pass x to something that takes Maybe String, you are telling the compiler that x has two different types at the same time, so the compiler reports a type error. From no.email at nospam.invalid Mon Feb 2 01:12:38 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 22:12:38 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <6eb91c4b-92ff-44a8-b5a9-6ef04c71f4cb@googlegroups.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87h9v43kq1.fsf@jester.gateway.sonic.net> Steven D'Aprano writes: > if type(ptr) == A: > if ptr != Anil: ... > if type(ptr) == B: > if ptr != Bnil: ... > etc. That would be insane. So how does Haskell do this? That wouldn't make sense in Haskell: the types are known at compile time, so you wouldn't do that runtime switching on them. > At the bottom of the page is a link to a .mov version. Didn't see that earlier. Managed to download and mplayer is able to show it. Thanks! You might like: http://learnyouahaskell.com/chapters From rosuav at gmail.com Mon Feb 2 01:16:08 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Feb 2015 17:16:08 +1100 Subject: Python is DOOMED! Again! In-Reply-To: <87lhkg3ky7.fsf@jester.gateway.sonic.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> <87lhkg3ky7.fsf@jester.gateway.sonic.net> Message-ID: On Mon, Feb 2, 2015 at 5:07 PM, Paul Rubin wrote: > Chris Angelico writes: >> So since you can set something to Nothing regardless of type, and >> compare it against Nothing regardless of type, it doesn't really much >> matter that there are different types of Nothing. Right? > > No that's not how type inference works. If you have x = Nothing and > pass it to a function that takes a Maybe Int, type inference means the > compiler figures out that x must have type Maybe Int. If you then also > pass x to something that takes Maybe String, you are telling the > compiler that x has two different types at the same time, so the > compiler reports a type error. If you say "x = 5" and pass it to a function that accepts "Int or String", the compiler knows that it's actually an Int. If you then also pass that x to something that takes "Int or List", is that legal? If so, then 5 is separate from the "or String" and "or List" parts, and Nothing is actually typed. If not, then it's x, not Nothing, that has the type. ChrisA From no.email at nospam.invalid Mon Feb 2 01:25:20 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 22:25:20 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> <87lhkg3ky7.fsf@jester.gateway.sonic.net> Message-ID: <87d25s3k4v.fsf@jester.gateway.sonic.net> Chris Angelico writes: > If you say "x = 5" and pass it to a function that accepts "Int or > String", the compiler knows that it's actually an Int. If you then > also pass that x to something that takes "Int or List", is that legal? You'd have to do that with type classes, but yeah, the compiler figures out that x is an Int. > If so, then 5 is separate from the "or String" and "or List" parts, > and Nothing is actually typed. If not, then it's x, not Nothing, that > has the type. Not sure what you mean there. Haskell is statically typed which means all expressions including literals have types. And an equality like x = y + z requires the two sides of the equality to have the same type. So if you say x = Nothing and the compiler infers (from some other place in the program) that x has type Maybe String, then the Nothing you wrote also has type Maybe String. From steve+comp.lang.python at pearwood.info Mon Feb 2 02:01:46 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 18:01:46 +1100 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf20db$0$11117$c3e8da3@news.astraweb.com> Chris Angelico wrote: > On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano > wrote: >> And there are underspecified rules too. What is the plural of octopus? No >> fair looking it up in the dictionary. > > Standard and well-known piece of trivia, and there are several > options. "Octopodes" is one of the most rigorously formal, but > "octopuses" is perfectly acceptable. "Octopi" is technically > incorrect, as the -us ending does not derive from the Latin. And that would be relevant if we were speaking Latin, but we aren't :-P -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 2 02:15:56 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 18:15:56 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> Gregory Ewing wrote: > Steven D'Aprano wrote: >> [quote] >> If the object has a method named __dir__(), this method will >> be called and must return the list of attributes. >> [end quote] >> >> The first inaccuracy is that like all (nearly all?) dunder methods, >> Python only looks for __dir__ on the class, not the instance itself. > > It says "method", not "attribute", so technically > it's correct. The methods of an object are defined > by what's in its class. Citation please. I'd like to see where that is defined. Even if it is so defined, the definition is wrong. You can define methods on an instance. I showed an example of an instance with its own personal __dir__ method, and showed that dir() ignores it if the instance belongs to a new-style class but uses it if it is an old-style class. -- Steven From steve+comp.lang.python at pearwood.info Mon Feb 2 02:18:48 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 18:18:48 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> <87lhkg3ky7.fsf@jester.gateway.sonic.net> Message-ID: <54cf24d9$0$12991$c3e8da3$5496439d@news.astraweb.com> Paul Rubin wrote: > Chris Angelico writes: >> So since you can set something to Nothing regardless of type, and >> compare it against Nothing regardless of type, it doesn't really much >> matter that there are different types of Nothing. Right? > > No that's not how type inference works. If you have x = Nothing and > pass it to a function that takes a Maybe Int, type inference means the > compiler figures out that x must have type Maybe Int. If you then also > pass x to something that takes Maybe String, you are telling the > compiler that x has two different types at the same time, so the > compiler reports a type error. No apples and no oranges aren't the same thing, but if somebody is expecting no apples, and I give them no oranges instead, it would be churlish for them to complain that none of them are the wrong kind of fruit. -- Steve From marko at pacujo.net Mon Feb 2 02:39:10 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 09:39:10 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> Message-ID: <87fvaoep9d.fsf@elektro.pacujo.net> Michael Torrie : > http://en.cppreference.com/w/cpp/utility/functional/function > > Thus if we were to shoehorn your example into C++, the result would be > idiomatically very similar to what you have in your Python code. I can understand why you wouldn't write out my example in C++: using std::placeholders::_1; std::function f_add_display2 = std::bind( &Foo::print_add, foo, _1 ); vs f_add_display2 = foo.print_add The cherry on top: "_1"! The C++ compiler figures out template types heroically but can't wrap its head around the arity of the method. Marko From jeanpierreda at gmail.com Mon Feb 2 02:41:31 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 1 Feb 2015 23:41:31 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: -- Devin On Sun, Feb 1, 2015 at 11:15 PM, Steven D'Aprano wrote: > Gregory Ewing wrote: > >> Steven D'Aprano wrote: >>> [quote] >>> If the object has a method named __dir__(), this method will >>> be called and must return the list of attributes. >>> [end quote] >>> >>> The first inaccuracy is that like all (nearly all?) dunder methods, >>> Python only looks for __dir__ on the class, not the instance itself. >> >> It says "method", not "attribute", so technically >> it's correct. The methods of an object are defined >> by what's in its class. > > Citation please. I'd like to see where that is defined. https://docs.python.org/3/glossary.html#term-method > Even if it is so defined, the definition is wrong. You can define methods on > an instance. I showed an example of an instance with its own personal > __dir__ method, and showed that dir() ignores it if the instance belongs to > a new-style class but uses it if it is an old-style class. You didn't define a method, you defined a callable attribute. Old-style classes will call those for special method overriding, because it's the simplest thing to do. New-style classes look methods up on the class as an optimization, but it also really complicates the attribute semantics. The lookup strategy is explicitly defined in the docs. pydoc is, like always, incomplete or inaccurate. See https://docs.python.org/2/reference/datamodel.html#special-method-names -- Devin From no.email at nospam.invalid Mon Feb 2 02:43:19 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 01 Feb 2015 23:43:19 -0800 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> <87lhkg3ky7.fsf@jester.gateway.sonic.net> <54cf24d9$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: <874mr43giw.fsf@jester.gateway.sonic.net> Steven D'Aprano writes: > No apples and no oranges aren't the same thing, but if somebody is expecting > no apples, and I give them no oranges instead, it would be churlish for them > to complain that none of them are the wrong kind of fruit. https://davedevine.wordpress.com/2011/01/20/the-sartre-joke/ From marko at pacujo.net Mon Feb 2 03:51:26 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 10:51:26 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <54cedb79$0$12981$c3e8da3$5496439d@news.astraweb.com> Message-ID: <874mr4k86p.fsf@elektro.pacujo.net> Steven D'Aprano : > Steven D'Aprano wrote: >> Of course people make grammar mistakes that they don't spot. > > Ironically, this is one of them. It should of course be "grammatical > mistakes". I don't believe you made a mistake according to your brain's grammar engine. Parenthetically, I don't believe you made a mistake even according to your English teacher. Marko From marko at pacujo.net Mon Feb 2 03:52:50 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 10:52:50 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87zj8witjx.fsf@elektro.pacujo.net> Chris Angelico : > On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano > wrote: >> And there are underspecified rules too. What is the plural of octopus? No >> fair looking it up in the dictionary. > > Standard and well-known piece of trivia, and there are several > options. "Octopodes" is one of the most rigorously formal, but > "octopuses" is perfectly acceptable. "Octopi" is technically > incorrect, as the -us ending does not derive from the Latin. Your brain's grammar engine will give you the correct answer. It may not match your English teacher's answer, but the language we are talking about is not standard English but the dialect you have acquired in childhood. Marko From yacinechaouche at yahoo.com Mon Feb 2 04:54:10 2015 From: yacinechaouche at yahoo.com (yacinechaouche at yahoo.com) Date: Mon, 2 Feb 2015 01:54:10 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? Message-ID: I wrote a little script that acts like a proxy, you just give it a URL and it will fetch the content and display it back to you. For some reason, this proxy blocks sometimes and refuses to serve any new queries. The script still runs, but it seems like it's stuck somewhere. When I strace it to see what it's doing, I find it hanging on this instruction : root at backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # strace -fp 6918 Process 6918 attached - interrupt to quit recvfrom(6, ^CProcess 6918 detached root at backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # I read in the SimpleHTTPServer source code that one can inherit from the SocketServer.TrheadingMixIn mixin to enable a threaded server to handle multiple requests at a time instead of just one (thinking maybe that's what was blocking it). However, it seems like it has nothing to do with my problem. What I need to do is not only handle multiple requests at a time, but more importantly to make the request handler non-blocking. Any ideas ? here's come code : import SimpleHTTPServer import BaseHTTPServer import SocketServer import requests class Handler(SocketServer.ThreadingMixIn,SimpleHTTPServer.SimpleHTTPRequestHandler): def do_GET(self): self.send_response(200) self.send_header('Content-Type', 'text/html') self.end_headers() # self.path will contain a URL to be fetched by my proxy self.wfile.write(getFlux(self.path.lstrip("/"))) session = requests.Session() IP,PORT = "MY_IP_HERE",8080 def getFlux(url): response = session.get(url) s = response.text return s server = BaseHTTPServer.HTTPServer((IP,PORT),Handler) server.serve_forever() Thank you. From amirouche.boubekki at gmail.com Mon Feb 2 05:07:05 2015 From: amirouche.boubekki at gmail.com (Amirouche Boubekki) Date: Mon, 02 Feb 2015 10:07:05 +0000 Subject: How to write a non blocking SimpleHTTPRequestHandler ? Message-ID: On Mon Feb 02 2015 at 10:55:26 AM wrote: > I wrote a little script that acts like a proxy, you just give it a URL and > it will fetch the content and display it back to you. > > For some reason, this proxy blocks sometimes and refuses to serve any new > queries. The script still runs, but it seems like it's stuck somewhere. > > When I strace it to see what it's doing, I find it hanging on this > instruction : > root at backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # strace -fp 6918 > Process 6918 attached - interrupt to quit > recvfrom(6, > ^CProcess 6918 detached > root at backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # > > I read in the SimpleHTTPServer source code that one can inherit from the > SocketServer.TrheadingMixIn mixin to enable a threaded server to handle > multiple requests at a time instead of just one (thinking maybe that's what > was blocking it). However, it seems like it has nothing to do with my > problem. What I need to do is not only handle multiple requests at a time, > but more importantly to make the request handler non-blocking. > > Any ideas ? here's come code : > > import SimpleHTTPServer > import BaseHTTPServer > import SocketServer > import requests > > class Handler(SocketServer.ThreadingMixIn,SimpleHTTPServer.SimpleH > TTPRequestHandler): > def do_GET(self): > self.send_response(200) > self.send_header('Content-Type', 'text/html') > self.end_headers() > # self.path will contain a URL to be fetched by my proxy > self.wfile.write(getFlux(self.path.lstrip("/"))) > > session = requests.Session() > IP,PORT = "MY_IP_HERE",8080 > > def getFlux(url): > response = session.get(url) > s = response.text > return s > > server = BaseHTTPServer.HTTPServer((IP,PORT),Handler) > server.serve_forever() > Your code seem perfectly fine. I had some trouble with py3's http.server with IE10 (in a virtualbox...), I put together a small server script similar to http.server that doesn't hang up on microsoft. It works with ayncio. It's not ready to serve big files, but hopefully you can fix that. HTH > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: server.py Type: text/x-python Size: 3455 bytes Desc: not available URL: From steve+comp.lang.python at pearwood.info Mon Feb 2 07:06:13 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 23:06:13 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > -- Devin > > On Sun, Feb 1, 2015 at 11:15 PM, Steven D'Aprano > wrote: >> Gregory Ewing wrote: >> >>> Steven D'Aprano wrote: >>>> [quote] >>>> If the object has a method named __dir__(), this method will >>>> be called and must return the list of attributes. >>>> [end quote] >>>> >>>> The first inaccuracy is that like all (nearly all?) dunder methods, >>>> Python only looks for __dir__ on the class, not the instance itself. >>> >>> It says "method", not "attribute", so technically >>> it's correct. The methods of an object are defined >>> by what's in its class. >> >> Citation please. I'd like to see where that is defined. > > https://docs.python.org/3/glossary.html#term-method Run this code using any version of Python from 1.5 onwards, and you will see that the definition is wrong: # === cut === class K: def f(self): pass # Define a function OUTSIDE of a class body. def g(self): pass K.g = g instance = K() assert type(instance.f) is type(instance.g) print(type(instance.f)) print(type(instance.g)) # === cut === Both K.f and K.g are methods, even though only one meets the definition given in the glossary. The glossary is wrong. Or rather, it is not so much that it is *wrong*, but that it is incomplete and over-simplified. It describes how methods are normally (but not always) defined, but not what they are. It is rather like defining "coffee" as the milky drink you buy from Starbucks, then insisting that the black liquid that you drank in an Italian restaurant cannot be coffee because you didn't buy it from Starbucks. Glossary entries are typically simplified, not exhaustive. It is not wise to take a three line glossary entry as a complete, definite explanation. In this case the glossary fails to tell you that methods are not *required* to be defined inside a class body, that is merely the usual way to do it. >> Even if it is so defined, the definition is wrong. You can define methods >> on an instance. I showed an example of an instance with its own personal >> __dir__ method, and showed that dir() ignores it if the instance belongs >> to a new-style class but uses it if it is an old-style class. > > You didn't define a method, you defined a callable attribute. That is wrong. I defined a method: py> from types import MethodType py> type(instance.f) is MethodType True instance.f is a method by the glossary definition. Its type is identical to types.MethodType, which is what I used to create a method by hand. I could also call the descriptor __get__ method by hand, if you prefer: py> def h(self): pass ... py> method = h.__get__(K, instance) py> assert type(method) is type(instance.f) py> print(method) > -- Steven From steve+comp.lang.python at pearwood.info Mon Feb 2 07:09:14 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 23:09:14 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf68eb$0$12996$c3e8da3$5496439d@news.astraweb.com> Steven D'Aprano wrote: > Both K.f and K.g are methods, even though only one meets the definition > given in the glossary. The glossary is wrong. Oh I'm sure somebody is going to pick me up on this... In Python 2, they are methods. In Python 3, they are functions, and aren't converted into methods until you access them via the instance: K.f returns the function f instance.f typically retrieves the function f from K, and converts it to a method object bound to instance -- Steven From rustompmody at gmail.com Mon Feb 2 07:12:35 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 2 Feb 2015 04:12:35 -0800 (PST) Subject: Python is DOOMED! Again! In-Reply-To: <874mr43giw.fsf@jester.gateway.sonic.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <8761bm40ud.fsf@jester.gateway.sonic.net> <54ce54d1$0$12989$c3e8da3$5496439d@news.astraweb.com> <87lhkg3ky7.fsf@jester.gateway.sonic.net> <54cf24d9$0$12991$c3e8da3$5496439d@news.astraweb.com> <874mr43giw.fsf@jester.gateway.sonic.net> Message-ID: On Monday, February 2, 2015 at 1:13:30 PM UTC+5:30, Paul Rubin wrote: > Steven D'Aprano writes: > > No apples and no oranges aren't the same thing, but if somebody is expecting > > no apples, and I give them no oranges instead, it would be churlish for them > > to complain that none of them are the wrong kind of fruit. > > https://davedevine.wordpress.com/2011/01/20/the-sartre-joke/ Actually the Sartre joke is more applicable to haskell than it might appear at first blush. li = [1,2,3] : [Int] -- a monomorphic type just as lc = ['a','b','c'] : [Char] lli = [[1,2],[3]] : [[Int]] However [] is a polymorphic value ie [] : [t] -- t is a type variable And now if we take tail (tail (tail li)) you get [] just as if you take tail (tail lli) However the two '[]-s' are of different types and so if you try to say append them you will get a Sartre error: The list of no integers is incompatible with the list of no lists of integers From rustompmody at gmail.com Mon Feb 2 07:27:29 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 2 Feb 2015 04:27:29 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <35a40ec6-3763-448b-9ea4-4a233a04979b@googlegroups.com> <54c1ccc8$0$13005$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: <29c73890-f2f2-4e81-8391-07ff5f1b1c7d@googlegroups.com> On Monday, February 2, 2015 at 10:57:27 AM UTC+5:30, Vito De Tullio wrote: > Steven D'Aprano wrote: > > > Checking the REPL first would have revealed that [].__dir__ raises > > AttributeError. In other words, lists don't have a __dir__ method. > > ? > > Python 3.4.2 (default, Nov 29 2014, 00:45:45) > [GCC 4.8.3] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> [].__dir__() > ['sort', '__contains__', '__init__', '__ge__', 'count', '__class__', > '__format__', '__mul__', 'index', '__rmul__', '__hash__', '__iter__', > 'clear', '__subclasshook__', '__getitem__', 'reverse', 'append', '__ne__', > 'pop', '__reduce__', '__add__', 'extend', '__gt__', '__sizeof__', > '__setattr__', '__imul__', '__dir__', '__le__', 'insert', '__repr__', > '__str__', '__getattribute__', '__len__', '__lt__', 'remove', '__new__', > '__reduce_ex__', 'copy', '__reversed__', '__delattr__', '__eq__', > '__setitem__', '__iadd__', '__doc__', '__delitem__'] > >>> Sure But as I said (and probably Steven checked): $ python Python 2.7.8 (default, Oct 20 2014, 15:05:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> [].__dir__ Traceback (most recent call last): File "", line 1, in AttributeError: 'list' object has no attribute '__dir__' --------------- My point was more methodological/sociological than technical: Are these dunder methods as 'internal' as say the register-allocation used by a C compiler? In which case the language implementer is entitled to tell the vanilla programmer: "Dunder methods (and their changingness) is none of your business" If however they are more on the public fa?ade of the language then some better docs would be nice From steve+comp.lang.python at pearwood.info Mon Feb 2 07:43:46 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 02 Feb 2015 23:43:46 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <29c73890-f2f2-4e81-8391-07ff5f1b1c7d@googlegroups.com> Message-ID: <54cf7104$0$12986$c3e8da3$5496439d@news.astraweb.com> Rustom Mody wrote: > My point was more methodological/sociological than technical: > > Are these dunder methods as 'internal' as say the register-allocation used > by a C compiler? Dunder methods are implementation, not interface. If you are the class author, then you care about the implementation, and write dunder methods. But as the class user, you should not call dunder methods directly, instead always go through the public interface: # Not these. a.__dir__() seq.__len__() x.__add__(y) spam.__eq__(ham) # Use these dir(a) len(seq) x + y spam == ham -- Steven From paul.nospam at rudin.co.uk Mon Feb 2 07:56:30 2015 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Mon, 02 Feb 2015 12:56:30 +0000 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> Message-ID: <86sieopj41.fsf@rudin.co.uk> Marko Rauhamaa writes: > Chris Angelico : > >> On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano >> wrote: >>> And there are underspecified rules too. What is the plural of octopus? No >>> fair looking it up in the dictionary. >> >> Standard and well-known piece of trivia, and there are several >> options. "Octopodes" is one of the most rigorously formal, but >> "octopuses" is perfectly acceptable. "Octopi" is technically >> incorrect, as the -us ending does not derive from the Latin. > > Your brain's grammar engine will give you the correct answer. It may not > match your English teacher's answer, but the language we are talking > about is not standard English but the dialect you have acquired in > childhood. Aha - the Humpty Dumpty approach to English usage: "When I use a word it means just what I choose it to mean..." From jeanpierreda at gmail.com Mon Feb 2 08:00:16 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 2 Feb 2015 05:00:16 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 4:06 AM, Steven D'Aprano wrote: >> On Sun, Feb 1, 2015 at 11:15 PM, Steven D'Aprano >> wrote: > Both K.f and K.g are methods, even though only one meets the definition > given in the glossary. The glossary is wrong. I agree, it oversimplified and has made a useless distinction here. >>> Even if it is so defined, the definition is wrong. You can define methods >>> on an instance. I showed an example of an instance with its own personal >>> __dir__ method, and showed that dir() ignores it if the instance belongs >>> to a new-style class but uses it if it is an old-style class. >> >> You didn't define a method, you defined a callable attribute. > > That is wrong. I defined a method: > > py> from types import MethodType > py> type(instance.f) is MethodType > True > > > instance.f is a method by the glossary definition. Its type is identical to > types.MethodType, which is what I used to create a method by hand. You are assuming that they are both methods, just because they are instances of a type called "MethodType". This is like assuming that a Tree() object is made out of wood. The documentation is free to define things in terms other than types and be correct. There are many properties of functions-on-classes that callable instance attributes that are instances of MethodType do not have, as we've already noticed. isinstance can say one thing, and the documentation another, and both can be right, because they are saying different things. For an example we can all agree on, this is not an instance of collections.Iterable, but the docs claim it is iterable: https://docs.python.org/2/glossary.html#term-iterable class MyIterable(object): def __getitem__(self, i): return i The docs are not "wrong", they are just making a distinction for humans that is separate from the python types involved. This is OK. -- Devin From jeanpierreda at gmail.com Mon Feb 2 08:02:39 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 2 Feb 2015 05:02:39 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c6d7c2$0$12992$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 5:00 AM, Devin Jeanpierre wrote: > On Mon, Feb 2, 2015 at 4:06 AM, Steven D'Aprano > wrote: >>> On Sun, Feb 1, 2015 at 11:15 PM, Steven D'Aprano >>> wrote: >> Both K.f and K.g are methods, even though only one meets the definition >> given in the glossary. The glossary is wrong. > > I agree, it oversimplified and has made a useless distinction here. Oops, I just realized why such a claim might be made: the documentation probably wants to be able to say that any method can use super(). So that's why it claims that it isn't a method unless it's defined inside a class body. -- Devin >>>> Even if it is so defined, the definition is wrong. You can define methods >>>> on an instance. I showed an example of an instance with its own personal >>>> __dir__ method, and showed that dir() ignores it if the instance belongs >>>> to a new-style class but uses it if it is an old-style class. >>> >>> You didn't define a method, you defined a callable attribute. >> >> That is wrong. I defined a method: >> >> py> from types import MethodType >> py> type(instance.f) is MethodType >> True >> >> >> instance.f is a method by the glossary definition. Its type is identical to >> types.MethodType, which is what I used to create a method by hand. > > You are assuming that they are both methods, just because they are > instances of a type called "MethodType". This is like assuming that a > Tree() object is made out of wood. > > The documentation is free to define things in terms other than types > and be correct. There are many properties of functions-on-classes that > callable instance attributes that are instances of MethodType do not > have, as we've already noticed. isinstance can say one thing, and the > documentation another, and both can be right, because they are saying > different things. > > > For an example we can all agree on, this is not an instance of > collections.Iterable, but the docs claim it is iterable: > https://docs.python.org/2/glossary.html#term-iterable > > class MyIterable(object): > def __getitem__(self, i): return i > > The docs are not "wrong", they are just making a distinction for > humans that is separate from the python types involved. This is OK. > > -- Devin From antoon.pardon at rece.vub.ac.be Mon Feb 2 08:20:56 2015 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Mon, 02 Feb 2015 14:20:56 +0100 Subject: Is there a cairo like surface for the screen without the window hassle Message-ID: <54CF79B8.10904@rece.vub.ac.be> I need to have a program construct a number of designs. Of course I can directly use a pfd surface and later use a pdf viewer to check. But that becomes rather cumbersome fast. But if I use a cairo-surface for on the screen I suddenly have to cope with expose events and all such things I am not really interested in. So does someone know of a package that provides a cairo like surface but that would take care of the events in a rather straight forward matter, so that my program could make it's design in a window on the screen just as if it is designing it in a pdf file. -- Antoon Pardon From marko at pacujo.net Mon Feb 2 08:27:00 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 15:27:00 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> <86sieopj41.fsf@rudin.co.uk> Message-ID: <87k300iguz.fsf@elektro.pacujo.net> Paul Rudin : > Marko Rauhamaa writes: >> Your brain's grammar engine will give you the correct answer. It may >> not match your English teacher's answer, but the language we are >> talking about is not standard English but the dialect you have >> acquired in childhood. > > Aha - the Humpty Dumpty approach to English usage: "When I use a word > it means just what I choose it to mean..." Yes, and your Humpty Dumpty brain is stringent about the proper usage. Your brain happens to be highly aligned with those of your childhood friends. Your communal dialect has thousands of rigorous rules, only you couldn't make a complete list of them. Marko From beliavsky at aol.com Mon Feb 2 08:30:56 2015 From: beliavsky at aol.com (beliavsky at aol.com) Date: Mon, 2 Feb 2015 05:30:56 -0800 (PST) Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> Message-ID: <1661614f-d021-4416-b81b-13db69e78d05@googlegroups.com> On Friday, January 30, 2015 at 5:51:38 PM UTC-5, Gregory Ewing wrote: > Michael Torrie wrote: > > On 01/30/2015 10:31 AM, Rustom Mody wrote: > > > >>And what about the grey area between lightweight and heavyweight? > > > > That's what the smart pointers are for. > > I'd say it's what higher-level languages are for. :-) > > I'm completely convinced nowadays that there is > *no* use case for C++. If you need to program the > bare metal, use C. For anything more complicated, > use a language that has proper memory-management > abstractions built in. Lots of people are using C++ to build packages for statistical programming language R, using the package Rcpp. It has been possible to build such packages for R and S using Fortran and C since the beginning, and many have done so, but the wide usage of Rcpp suggests that there are advantages to using C++. C++ is still the primary language used by financial derivatives quants. From jessemuzziddu at mail.com Mon Feb 2 09:03:19 2015 From: jessemuzziddu at mail.com (TROIA BERLUSCONIANA MARIA ELENA BOSCHI IN GALERA) Date: Mon, 2 Feb 2015 06:03:19 -0800 (PST) Subject: -- redacted -- Message-ID: <04938df1-8d59-4262-acff-648c1475d0f5@googlegroups.com> -- redacted -- From jessemuzziddu at mail.com Mon Feb 2 09:06:53 2015 From: jessemuzziddu at mail.com (TROIA BERLUSCONIANA MARIA ELENA BOSCHI IN GALERA) Date: Mon, 2 Feb 2015 06:06:53 -0800 (PST) Subject: -- redacted -- In-Reply-To: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: -- redacted -- From steve+comp.lang.python at pearwood.info Mon Feb 2 09:07:24 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 03 Feb 2015 01:07:24 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > On Mon, Feb 2, 2015 at 4:06 AM, Steven D'Aprano > wrote: >> instance.f is a method by the glossary definition. Its type is identical >> to types.MethodType, which is what I used to create a method by hand. > > You are assuming that they are both methods, just because they are > instances of a type called "MethodType". This is like assuming that a > Tree() object is made out of wood. No. It is "assuming" that a Tree() object is a Tree() object. Run this code: # === cut === class K(object): def f(self): pass def f(self): pass instance = K() things = [instance.f, f.__get__(instance, K)] from random import shuffle shuffle(things) print(things) # === cut === You allege that one of these things is a method, and the other is not. I challenge you to find any behavioural or functional difference between the two. (Object IDs don't count.) If you can find any meaningful difference between the two, I will accept that methods have to be created as functions inside a class body. Otherwise you are reduced to claiming that there is some sort of mystical, undetectable "essence" or "spirit" that makes one of those two objects a real method and the other one a fake method, even though they have the same type, the same behaviour, and there is no test that can tell you which is which. > The documentation is free to define things in terms other than types > and be correct. If you wanted to argue that "method" is a generic term, and we have instance methods, class methods, static methods, and any other sort of method we care to create using the descriptor protocol, then I would agree you have a point. But since we're just talking about instance methods, Python doesn't care how they came into existence. You can use def to create a function inside a class body, inject a function into the class, call the descriptor __get__ method, or use the types.MethodType type object, it is all the same. You can use a def statement, or a lambda, or types.FunctionType if you are really keen. It makes no difference. Do I expect the glossary to go into such pedantic detail? No, of course not. But I do expect anyone with a few years of Python programming experience to be able to understand that what makes a method be a method is its type and behaviour, not where it came from. > There are many properties of functions-on-classes that > callable instance attributes that are instances of MethodType do not > have, as we've already noticed. isinstance can say one thing, and the > documentation another, and both can be right, because they are saying > different things. > > > For an example we can all agree on, this is not an instance of > collections.Iterable, but the docs claim it is iterable: > https://docs.python.org/2/glossary.html#term-iterable > > class MyIterable(object): > def __getitem__(self, i): return i "Iterable" is a generic term, not a type. Despite the existence of the collections.Iterable ABC, "iterable" refers to any type which can be iterated over, using either of two different protocols. As I said above, if you wanted to argue that "method" was a general term for any callable attached to an instance or class, then you might have a point. But you're doing something much weirder: you are arguing that given two objects which are *identical* save for their object ID, one might be called a method, and the other not, due solely to where it was created. Not even where it was retrieved from, but where it was created. If you believe that "method or not" depends on where the function was defined, then this will really freak you out: py> class Q: ... def f(self): pass # f defined inside the class ... py> def f(self): pass # f defined outside the class ... py> f, Q.f = Q.f, f # Swap the "inside" f and the "outside" f. py> instance = Q() py> instance.f # Uses "outside" f, so not a real method! > py> MethodType(f, instance) # Uses "inside" f, so is a real method! > -- Steven From steve+comp.lang.python at pearwood.info Mon Feb 2 09:20:39 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 03 Feb 2015 01:20:39 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cf87b8$0$12991$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > Oops, I just realized why such a claim might be made: the > documentation probably wants to be able to say that any method can use > super(). So that's why it claims that it isn't a method unless it's > defined inside a class body. You can use super anywhere, including outside of classes. The only thing you can't do is use the Python 3 "super hack" which automatically fills in the arguments to super if you don't supply them. That is compiler magic which truly does require the function to be defined inside a class body. But you can use super outside of classes: py> class A(list): ... pass ... py> x = super(A) # Unbound super object! py> x.__get__(A).append py> a = A() py> x.__get__(a).append -- Steven From ginobusciarello at outlook.com Mon Feb 2 09:30:02 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Mon, 2 Feb 2015 06:30:02 -0800 (PST) Subject: -- redacted -- In-Reply-To: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> References: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> Message-ID: -- redacted -- From gabrielbianchiferreira at gmail.com Mon Feb 2 11:00:12 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 08:00:12 -0800 (PST) Subject: Downloading videos (in flash applications) using python Message-ID: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> Hello, I need some help in downloading videos from flash applications in web using python. Is there any lib to deal with flash player using python? The videos I need to download are, in fact, live streaming content. You can see an example here: http://vejoaovivo.com.br/sc/itapema/avenida-nereu-ramos-1029.html Thanks From breamoreboy at yahoo.co.uk Mon Feb 2 11:06:50 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 02 Feb 2015 16:06:50 +0000 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 02/02/2015 04:47, Chris Angelico wrote: > On Mon, Feb 2, 2015 at 3:14 PM, Roy Smith wrote: >> In article <54ceda0b$0$12977$c3e8da3$5496439d at news.astraweb.com>, >> Steven D'Aprano wrote: >> >>> What is the plural of octopus? >> >> It's a trick question. Octopus is already plural. Monopus is singular. > > People is already plural, too, but you can talk about all the peoples > of the world. Also, I can use "people" as the subject and "is" as the > verb, just to completely destroy any chance of a simple grammar parser > being able to cope with English. > > ChrisA > I'm simple and I cope (somehow) with English. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Mon Feb 2 11:08:41 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 02 Feb 2015 16:08:41 +0000 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <87zj8witjx.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> Message-ID: On 02/02/2015 08:52, Marko Rauhamaa wrote: > Chris Angelico : > >> On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano >> wrote: >>> And there are underspecified rules too. What is the plural of octopus? No >>> fair looking it up in the dictionary. >> >> Standard and well-known piece of trivia, and there are several >> options. "Octopodes" is one of the most rigorously formal, but >> "octopuses" is perfectly acceptable. "Octopi" is technically >> incorrect, as the -us ending does not derive from the Latin. > > Your brain's grammar engine will give you the correct answer. It may not > match your English teacher's answer, but the language we are talking > about is not standard English but the dialect you have acquired in > childhood. > > > Marko > I'd love to see a formal definition for "standard English". -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From steve+comp.lang.python at pearwood.info Mon Feb 2 11:19:35 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 03 Feb 2015 03:19:35 +1100 Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> Message-ID: <54cfa398$0$12984$c3e8da3$5496439d@news.astraweb.com> Gabriel Ferreira wrote: > Hello, > > I need some help in downloading videos from flash applications in web > using python. Is there any lib to deal with flash player using python? Try Youtube-DL http://rg3.github.io/youtube-dl -- Steven From rustompmody at gmail.com Mon Feb 2 11:21:47 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 2 Feb 2015 08:21:47 -0800 (PST) Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> Message-ID: <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> On Monday, February 2, 2015 at 9:40:35 PM UTC+5:30, Mark Lawrence wrote: > On 02/02/2015 08:52, Marko Rauhamaa wrote: > > Chris Angelico : > > > >> On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano wrote: > >>> And there are underspecified rules too. What is the plural of octopus? No > >>> fair looking it up in the dictionary. > >> > >> Standard and well-known piece of trivia, and there are several > >> options. "Octopodes" is one of the most rigorously formal, but > >> "octopuses" is perfectly acceptable. "Octopi" is technically > >> incorrect, as the -us ending does not derive from the Latin. > > > > Your brain's grammar engine will give you the correct answer. It may not > > match your English teacher's answer, but the language we are talking > > about is not standard English but the dialect you have acquired in > > childhood. > > > > > > Marko > > > > I'd love to see a formal definition for "standard English". > I'd also love to see a formal definition of 'formal' Elsewhere someone (Marko I think) used the term 'rigorous' Ive heard it said that formal is more rigorous than 'rigorous'. And of course the other way round as well ;-) From gabrielbianchiferreira at gmail.com Mon Feb 2 11:22:15 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 08:22:15 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: <54cfa398$0$12984$c3e8da3$5496439d@news.astraweb.com> References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <54cfa398$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: Thanks for the Hint... But it seems not to support the website i mentioned... Is there a way to make it possible for any kind of video player in the net? Flash player... Since it's not a ordinary video... I'm trying to record & download a live streaming video!! Thx From torriem at gmail.com Mon Feb 2 11:25:32 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 02 Feb 2015 09:25:32 -0700 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <87fvaoep9d.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> <87fvaoep9d.fsf@elektro.pacujo.net> Message-ID: <54CFA4FC.1090100@gmail.com> On 02/02/2015 12:39 AM, Marko Rauhamaa wrote: > Michael Torrie : > >> http://en.cppreference.com/w/cpp/utility/functional/function >> >> Thus if we were to shoehorn your example into C++, the result would be >> idiomatically very similar to what you have in your Python code. > > I can understand why you wouldn't write out my example in C++: I wouldn't write it out because it's a contrived example, and why should I waste my time? You have no intention of being impressed with C++, let alone simply learn about it. > > using std::placeholders::_1; > > std::function f_add_display2 = > std::bind( &Foo::print_add, foo, _1 ); Looks okay to me. That's normal C++ that would be clear to any C++ programmer. And for a statically compiled language, that's incredibly powerful while providing for a measure of safety. You may shudder, but it's a really good statically compiled solution, given the constraints of C++ and its compiler. But like I said, idiomatically in C++, this isn't normally what you'd do anyway. You'd store a std::ostream object. > vs > > f_add_display2 = foo.print_add > > The cherry on top: "_1"! The C++ compiler figures out template types > heroically but can't wrap its head around the arity of the method. std::bind is a generic function that works with undecorated function pointers. There's no possible way for the compiler to know the arity of the function without you telling it (how else would you do it? I honestly want to know.). That you would find this fact to be incredulous suggests that you have very little understanding of compilers in general. Python being a dynamic, interpreted language can examine a function at runtime. Python is a very expressive language compared to C++ that can do things at runtime that C++ cannot. It's also an interpreted, dynamically-typed language, whereas C++ is a statically-typed language that is clearly faster at many things than Python is. It's a matter of different tools for different jobs. For most of my jobs Python is the right tool. But I can see why people would still want to pick a compiled language, and I can understand why some people still use C++. It's a very powerful language. I'm a little unclear as to why you're even bringing up the comparison with C++. What's your point? To feel superior? From breamoreboy at yahoo.co.uk Mon Feb 2 11:52:03 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 02 Feb 2015 16:52:03 +0000 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> References: <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> Message-ID: On 02/02/2015 16:21, Rustom Mody wrote: > On Monday, February 2, 2015 at 9:40:35 PM UTC+5:30, Mark Lawrence wrote: >> On 02/02/2015 08:52, Marko Rauhamaa wrote: >>> Chris Angelico : >>> >>>> On Mon, Feb 2, 2015 at 12:59 PM, Steven D'Aprano wrote: >>>>> And there are underspecified rules too. What is the plural of octopus? No >>>>> fair looking it up in the dictionary. >>>> >>>> Standard and well-known piece of trivia, and there are several >>>> options. "Octopodes" is one of the most rigorously formal, but >>>> "octopuses" is perfectly acceptable. "Octopi" is technically >>>> incorrect, as the -us ending does not derive from the Latin. >>> >>> Your brain's grammar engine will give you the correct answer. It may not >>> match your English teacher's answer, but the language we are talking >>> about is not standard English but the dialect you have acquired in >>> childhood. >>> >>> >>> Marko >>> >> >> I'd love to see a formal definition for "standard English". >> > > I'd also love to see a formal definition of 'formal' > Elsewhere someone (Marko I think) used the term 'rigorous' > > Ive heard it said that formal is more rigorous than 'rigorous'. > And of course the other way round as well ;-) > I'd like to see anybody define 'a' and 'the' without using 'a' and 'the'. Would that be formally rigorous or rigorously formal? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From no.email at nospam.invalid Mon Feb 2 11:54:04 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 02 Feb 2015 08:54:04 -0800 Subject: Is there a cairo like surface for the screen without the window hassle References: Message-ID: <87zj8w1cgj.fsf@jester.gateway.sonic.net> Antoon Pardon writes: > So does someone know of a package that provides a cairo like surface Not sure what a cairo like surface is, but maybe you want HTML5 Canvas that's built into recent browsers. So you'd just embed a web server in your application and interact with it through a browser. From rosuav at gmail.com Mon Feb 2 12:25:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 04:25:58 +1100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> Message-ID: On Tue, Feb 3, 2015 at 3:52 AM, Mark Lawrence wrote: > I'd like to see anybody define 'a' and 'the' without using 'a' and 'the'. > Would that be formally rigorous or rigorously formal? a: Indefinite article, used to represent individual objects not otherwise identifiable. the: Definite article, used to represent individual objects identifiable by context. Near enough? ChrisA From marko at pacujo.net Mon Feb 2 12:57:44 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 19:57:44 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> <87fvaoep9d.fsf@elektro.pacujo.net> Message-ID: <8761bkdwmf.fsf@elektro.pacujo.net> Michael Torrie : > You have no intention of being impressed with C++, let alone simply > learn about it. I am fully open to being impressed. I have more than a decade of C++ programming under my belt, although not much for the past few years. > There's no possible way for the compiler to know the arity of > the function without you telling it (how else would you do it? Somehow, C# manages it just fine. > I honestly want to know.). See > That you would find this fact to be incredulous suggests that you have > very little understanding of compilers in general. Python being a > dynamic, interpreted language can examine a function at runtime. C# does it at compile-time. C++ could have and should have introduced delegates early on. The method pointer syntax was screaming for delegate semantics. Too bad that didn't occur to Stroustrup until it was too late. > I'm a little unclear as to why you're even bringing up the comparison > with C++. What's your point? To feel superior? I really don't understand why you are taking all of this so personally. We are just discussing different aspects of different programming languages. Marko From torriem at gmail.com Mon Feb 2 13:23:44 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 02 Feb 2015 11:23:44 -0700 Subject: [OT] fortran lib which provide python like data type In-Reply-To: <8761bkdwmf.fsf@elektro.pacujo.net> References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> <87fvaoep9d.fsf@elektro.pacujo.net> <8761bkdwmf.fsf@elektro.pacujo.net> Message-ID: <54CFC0B0.5020006@gmail.com> On 02/02/2015 10:57 AM, Marko Rauhamaa wrote: > I really don't understand why you are taking all of this so personally. > We are just discussing different aspects of different programming > languages. Fair enough. You raise good points. I am not taking it personally; your emails, lacking emotional context, just seemed a bit unnecessarily argumentative. For example, "The cherry on top: "_1"! The C++ compiler figures out template types heroically but can't wrap its head around the arity of the method". From breamoreboy at yahoo.co.uk Mon Feb 2 13:34:42 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 02 Feb 2015 18:34:42 +0000 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> Message-ID: On 02/02/2015 17:25, Chris Angelico wrote: > On Tue, Feb 3, 2015 at 3:52 AM, Mark Lawrence wrote: >> I'd like to see anybody define 'a' and 'the' without using 'a' and 'the'. >> Would that be formally rigorous or rigorously formal? > > a: Indefinite article, used to represent individual objects not > otherwise identifiable. > > the: Definite article, used to represent individual objects > identifiable by context. > > Near enough? > > ChrisA > Nope. 'article' begins with 'a' so you can't use it to define itself. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From irmen.NOSPAM at xs4all.nl Mon Feb 2 13:43:38 2015 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 02 Feb 2015 19:43:38 +0100 Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <54cfa398$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54cfc55b$0$2887$e4fe514c@news.xs4all.nl> On 2-2-2015 17:22, Gabriel Ferreira wrote: > Thanks for the Hint... But it seems not to support the website i mentioned... Is > there a way to make it possible for any kind of video player in the net? Flash > player... Since it's not a ordinary video... I'm trying to record & download a live > streaming video!! > > Thx > Youtube-dl has a modular structure and is open-source: https://github.com/rg3/youtube-dl/ It should be fairly easy to add a module/plugin that groks your site. Irmen From irmen.NOSPAM at xs4all.nl Mon Feb 2 13:47:55 2015 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 02 Feb 2015 19:47:55 +0100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: Message-ID: <54cfc65c$0$2887$e4fe514c@news.xs4all.nl> On 2-2-2015 10:54, yacinechaouche at yahoo.com wrote: > I wrote a little script that acts like a proxy, you just give it a URL and it will > fetch the content and display it back to you. > > For some reason, this proxy blocks sometimes and refuses to serve any new queries. > The script still runs, but it seems like it's stuck somewhere. > > When I strace it to see what it's doing, I find it hanging on this instruction : > root at backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # strace -fp 6918 Process 6918 attached > - interrupt to quit recvfrom(6, ^CProcess 6918 detached root at backup[10.10.10.21] > ~/SCRIPTS/INFOMANIAK # > > I read in the SimpleHTTPServer source code that one can inherit from the > SocketServer.TrheadingMixIn mixin to enable a threaded server to handle multiple > requests at a time instead of just one (thinking maybe that's what was blocking it). > However, it seems like it has nothing to do with my problem. What I need to do is not > only handle multiple requests at a time, but more importantly to make the request > handler non-blocking. Why? If you have multiple threads serving some requests at the same time, doesn't that already achieve your goal? In other words, have you tried what you describe above? (make sure you close the connection correctly or you'll be hogging a thread which may eventually make the server non responsive) Irmen From rajanbond at gmail.com Mon Feb 2 13:48:00 2015 From: rajanbond at gmail.com (rajanbond at gmail.com) Date: Mon, 2 Feb 2015 10:48:00 -0800 (PST) Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: On Saturday, 31 January 2015 18:39:01 UTC-8, Jason Friedman wrote: > > I have two lists > > > > l1 = ["a","b","c","d","e","f","g","h","i","j"] > > l2 = ["aR","bR","cR"] > > > > l2 will always be smaller or equal to l1 > > > > numL1PerL2 = len(l1)/len(l2) > > > > I want to create a dictionary that has key from l1 and value from l2 based on numL1PerL2 > > > > So > > > > { > > a:aR, > > b:aR, > > c:aR, > > d:bR, > > e:bR, > > f:bR, > > g:cR, > > h:cR, > > i:cR, > > j:cR > > } > > Another possibility is: > import itertools > my_dict = {x:y for x,y in zip(list1, itertools.cycle(list2))} NO. Sorry if this was not very clear. In teh above example- len(l1) = 10 len(l2) = 3 So, the dict can not have more than 3 keys from l1 with same value from l2 except the last element from l1 . So a,b,c will have one key- say aR d,e,f - bR g,h,i,j- cR- j has key cR because the number l1 is not completely divisible by l2 and leave a remainder. From marko at pacujo.net Mon Feb 2 13:54:51 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Feb 2015 20:54:51 +0200 Subject: [OT] fortran lib which provide python like data type References: <4822a519-5814-433b-88ad-9bea8a2f83e3@googlegroups.com> <3c31aec4-0c0a-4eb9-aff4-e7b7aa76da21@googlegroups.com> <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <871tma406m.fsf@jester.gateway.sonic.net> <8761bmf4gn.fsf@elektro.pacujo.net> <87oapde98u.fsf@elektro.pacujo.net> <87fvaoep9d.fsf@elektro.pacujo.net> <8761bkdwmf.fsf@elektro.pacujo.net> Message-ID: <871tm8dtz8.fsf@elektro.pacujo.net> Michael Torrie : > Fair enough. You raise good points. I am not taking it personally; your > emails, lacking emotional context, just seemed a bit unnecessarily > argumentative. For example, "The cherry on top: "_1"! The C++ compiler > figures out template types heroically but can't wrap its head around the > arity of the method". I feel programming languages, being inanimate abstractions, can take my abuse. Marko From ben+python at benfinney.id.au Mon Feb 2 13:57:24 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 03 Feb 2015 05:57:24 +1100 Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> Message-ID: <85sieogmzv.fsf@benfinney.id.au> Gabriel Ferreira writes: > I need some help in downloading videos from flash applications in web > using python. Is there any lib to deal with flash player using python? Yes, the ?youtube-dl? application is the most likely to have the library code you need. > The videos I need to download are, in fact, live streaming content. > You can see an example here: > > http://vejoaovivo.com.br/sc/itapema/avenida-nereu-ramos-1029.html If ?youtube-dl? doesn't support a particular site, please report a bug at its bug tracker. I'm sure they would like to have a working implementation too. This isn't really a question about using Python, so it may be best to continue the discussion in a bug report against ?youtube-dl?. -- \ ?It is far better to grasp the universe as it really is than to | `\ persist in delusion, however satisfying and reassuring.? ?Carl | _o__) Sagan | Ben Finney From rajanbond at gmail.com Mon Feb 2 14:00:36 2015 From: rajanbond at gmail.com (rajanbond at gmail.com) Date: Mon, 2 Feb 2015 11:00:36 -0800 (PST) Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: <8c614bc1-4497-4406-8fb2-0c0034defd42@googlegroups.com> On Saturday, 31 January 2015 18:39:01 UTC-8, Jason Friedman wrote: > > I have two lists > > > > l1 = ["a","b","c","d","e","f","g","h","i","j"] > > l2 = ["aR","bR","cR"] > > > > l2 will always be smaller or equal to l1 > > > > numL1PerL2 = len(l1)/len(l2) > > > > I want to create a dictionary that has key from l1 and value from l2 based on numL1PerL2 > > > > So > > > > { > > a:aR, > > b:aR, > > c:aR, > > d:bR, > > e:bR, > > f:bR, > > g:cR, > > h:cR, > > i:cR, > > j:cR > > } > > Another possibility is: > import itertools > my_dict = {x:y for x,y in zip(list1, itertools.cycle(list2))} Thank you Jason! Looks like this will work for my case. I had a solution working with count and num to keep track of the number of times l2 items are assigned as value but was looking for a more pythonic way. count = 0 num = 0 map = {} # for item in l2: # while count < numL1PerL2 and num <= len(l1): # map[l1[num]] = item # count += 1 # num += 1 # count = 0 # map[l1[num]] = l2[-1] From gabrielbianchiferreira at gmail.com Mon Feb 2 14:15:45 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 11:15:45 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> Message-ID: <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> I appreciate your help. I'm just afraid that Youtube-DL doesn't allow me to record or download a LIVE STREAMING VIDEO. Do you guys think it is possible, since I make some adjustments into the code of the library? From no.email at nospam.invalid Mon Feb 2 14:24:10 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 02 Feb 2015 11:24:10 -0800 Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> Message-ID: <87vbjk15id.fsf@jester.gateway.sonic.net> Gabriel Ferreira writes: > I appreciate your help. I'm just afraid that Youtube-DL doesn't allow > me to record or download a LIVE STREAMING VIDEO. Do you guys think it > is possible, since I make some adjustments into the code of the > library? If the stream operator really wants stop you from downloading (and some of them do), they can make it pretty difficult. I haven't looked at youtube-dl closely but those things are generally http retrievers. Flash videos can use other sorts of streaming protocols, with Macromedia or other DRM in some cases. You can presumably still get around it with enough hassle, but I decided a long time ago that I'd rather just shrink the audience of those sorts of streams, by choosing not to watch them. From gabrielbianchiferreira at gmail.com Mon Feb 2 14:30:07 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 11:30:07 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: <87vbjk15id.fsf@jester.gateway.sonic.net> References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> Message-ID: Hi Paul, I presume the stream operator doesn't want to prevent me from downloading or recording the videos. I just wanna know more about some lib that could be used to deal with Flash Player Applications... Or possibly, anything that could lead me to be able to get those streaming videos. Thx From irmen.NOSPAM at xs4all.nl Mon Feb 2 15:01:28 2015 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 02 Feb 2015 21:01:28 +0100 Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> Message-ID: <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> On 2-2-2015 20:30, Gabriel Ferreira wrote: > Hi Paul, I presume the stream operator doesn't want to prevent me from downloading or recording the videos. I just wanna know more about some lib that could be used to deal with Flash Player Applications... Or possibly, anything that could lead me to be able to get those streaming videos. > > Thx > Right, have you tried VLC then? (unless you really want to do this from python...) VLC can play streams, probably yours too, and convert it to another file. Irmen From no.email at nospam.invalid Mon Feb 2 15:16:12 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 02 Feb 2015 12:16:12 -0800 Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> Message-ID: <87r3u8133n.fsf@jester.gateway.sonic.net> Gabriel Ferreira writes: > Hi Paul, I presume the stream operator doesn't want to prevent me from > downloading or recording the videos. It's hard to tell why anyone uses Flash anymore, but impeding (if not preventing) downloads is probably a common reason. > I just wanna know more about some lib that could be used to deal with > Flash Player Applications... Or possibly, anything that could lead me > to be able to get those streaming videos. Gnash maybe? https://www.gnu.org/software/gnash/ From torriem at gmail.com Mon Feb 2 16:26:02 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 02 Feb 2015 14:26:02 -0700 Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> Message-ID: <54CFEB6A.6020200@gmail.com> On 02/02/2015 12:30 PM, Gabriel Ferreira wrote: > Hi Paul, I presume the stream operator doesn't want to prevent me > from downloading or recording the videos. I just wanna know more > about some lib that could be used to deal with Flash Player > Applications... Or possibly, anything that could lead me to be able > to get those streaming videos. You can't do it directly with any Python library that I know of. You can, however, use the tools that come with rtmpdump: https://rtmpdump.mplayerhq.hu/ It's not automatic, and requires some iptables tricks, but once you get the stream information, you can always download it via rtmpdump. From gabrielbianchiferreira at gmail.com Mon Feb 2 17:09:56 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 14:09:56 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> Message-ID: <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> The original plan was doing it using python. But, anyway, I should build a robot program to do that task for me... Because I need to capture hundreds of hours of different videos... You mean that is difficult because it's Python... or it's difficult because it's a Flash Application? Thx From breamoreboy at yahoo.co.uk Mon Feb 2 17:13:03 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 02 Feb 2015 22:13:03 +0000 Subject: Downloading videos (in flash applications) using python In-Reply-To: <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> Message-ID: On 02/02/2015 22:09, Gabriel Ferreira wrote: > The original plan was doing it using python. But, anyway, I should build a robot program to do that task for me... Because I need to capture hundreds of hours of different videos... You mean that is difficult because it's Python... or it's difficult because it's a Flash Application? > > Thx > I don't actually know, but could you please provide some context and write in plain English, those damn ... things are extremely annoying. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Mon Feb 2 18:15:15 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 10:15:15 +1100 Subject: [OT] fortran lib which provide python like data type In-Reply-To: References: <3d3ef587-a62d-4024-964c-0ac8be023ca3@googlegroups.com> <0b8db0f6-75c5-4a8f-a9a5-51fa936b306e@googlegroups.com> <87zj8zf8bv.fsf@elektro.pacujo.net> <87bnlekfrg.fsf@elektro.pacujo.net> <561ab49b-890e-4c95-9cf2-6cbbac5b3c93@googlegroups.com> <87a90yf5dd.fsf@elektro.pacujo.net> <54ce5929$0$12992$c3e8da3$5496439d@news.astraweb.com> <87twz5e9yl.fsf@elektro.pacujo.net> <54ceda0b$0$12977$c3e8da3$5496439d@news.astraweb.com> <87zj8witjx.fsf@elektro.pacujo.net> <243defb2-b3b0-413d-adf6-67d1e8ecd7d5@googlegroups.com> Message-ID: On Tue, Feb 3, 2015 at 5:34 AM, Mark Lawrence wrote: > On 02/02/2015 17:25, Chris Angelico wrote: >> >> On Tue, Feb 3, 2015 at 3:52 AM, Mark Lawrence >> wrote: >>> >>> I'd like to see anybody define 'a' and 'the' without using 'a' and 'the'. >>> Would that be formally rigorous or rigorously formal? >> >> >> a: Indefinite article, used to represent individual objects not >> otherwise identifiable. >> >> Near enough? >> > > Nope. 'article' begins with 'a' so you can't use it to define itself. Hrm. This is difficult. a: Indefinite identifier, used to represent non-specific objects. Not a perfectly accurate definition now, but hey, I avoided the self-reference! ChrisA From gabrielbianchiferreira at gmail.com Mon Feb 2 18:21:07 2015 From: gabrielbianchiferreira at gmail.com (Gabriel Ferreira) Date: Mon, 2 Feb 2015 15:21:07 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> Message-ID: Mark Lawrence wrote: > I don't actually know, but could you please provide some context and > write in plain English, those damn ... things are extremely annoying. > Hi, Mark. I am developing a research project, which includes video analysis (computer vision, big data, data mining, etc). The first part of the project is about building a database containing a big amount of video of urban zones. I live in Brazil, so the plan is collecting videos from urban areas of Brazilian big cities, like Sao Paulo. I have to make this task "automatic". in other words, I don't want to manually download hundreds or thousands hours of videos. So I have to develop a "robot" to do that assingment for me. (I wish I could do that using Python). I have found a good website that provides me live images from several cities in Brazil. So that's it! That's what I need. I was expecting to develop a program to record & download those videos, that are being broadcasted live on that website. So I would be able to form my database and continue my research. The problem is that this particular website uses a flash player application do broadcast the live images of the urban areas. I'm not too familiar with Flash Applications. I don't know how to deal with them using Python. I was wondering if someone could help me solve this problem. The goal is recording & downloading those videos, from the mentioned website. And, yes, it has to be an automated program (no manual work). I wish I had been clear with you guys! THX! From nieko444 at gmail.com Mon Feb 2 18:26:32 2015 From: nieko444 at gmail.com (Luke Lloyd) Date: Mon, 2 Feb 2015 23:26:32 +0000 Subject: __pycache__ Message-ID: I am in school and there is a problem with my code: When I try to run my code in the python code in the python shell it waits about 10 seconds then shows an error that says ?IDLE?s subprocess didn?t make connection. Either IDLE can?t start a subprocess or personal firewall software is blocking the connection.? Then when I press OK it just closes. P.S: My code is encrypting and decrypting using an offset factor. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Mon Feb 2 20:53:08 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 03 Feb 2015 12:53:08 +1100 Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <87r3u8133n.fsf@jester.gateway.sonic.net> Message-ID: <54d02a05$0$12995$c3e8da3$5496439d@news.astraweb.com> Paul Rubin wrote: > It's hard to tell why anyone uses Flash anymore Masochism and/or sadism, depending on whether they get more pain from writing the Flash code in the first place or pleasure from forcing Flash on the viewer. -- Steven From rustompmody at gmail.com Mon Feb 2 21:41:09 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 2 Feb 2015 18:41:09 -0800 (PST) Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> Message-ID: <3f6eb480-32d5-4837-ac13-8508ee228a3f@googlegroups.com> On Tuesday, February 3, 2015 at 4:51:18 AM UTC+5:30, Gabriel Ferreira wrote: > Mark Lawrence wrote: > > > I don't actually know, but could you please provide some context and > > write in plain English, those damn ... things are extremely annoying. > > > > Hi, Mark. > > I am developing a research project, which includes video analysis (computer vision, big data, data mining, etc). The first part of the project is about building a database containing a big amount of video of urban zones. I live in Brazil, so the plan is collecting videos from urban areas of Brazilian big cities, like Sao Paulo. > > I have to make this task "automatic". in other words, I don't want to manually download hundreds or thousands hours of videos. So I have to develop a "robot" to do that assingment for me. (I wish I could do that using Python). Develop robot -- right Do it in python -- probably not so right > > I have found a good website that provides me live images from several cities in Brazil. So that's it! That's what I need. I was expecting to develop a program to record & download those videos, that are being broadcasted live on that website. So I would be able to form my database and continue my research. > > The problem is that this particular website uses a flash player application do broadcast the live images of the urban areas. I'm not too familiar with Flash Applications. I don't know how to deal with them using Python. I was wondering if someone could help me solve this problem. The goal is recording & downloading those videos, from the mentioned website. And, yes, it has to be an automated program (no manual work). The instinct is natural but probably inaccurate: Python is nice; JS/Flash not so much. However to get on top of this you probably need to debug the streaming with a modern browser using JS developer-tools/firebug etc, so that then you can script what you have debugged. If you do it in JS, there will be minor glitches once you have debugged the process. If you do it in python, you will have to jump into (rather out of) many hoops to go from JS to python. And debugging the process is not an option. [Disclaimer: I dont know what I am talking about] From torriem at gmail.com Mon Feb 2 22:12:53 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 02 Feb 2015 20:12:53 -0700 Subject: Downloading videos (in flash applications) using python In-Reply-To: References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> Message-ID: <54D03CB5.3090608@gmail.com> On 02/02/2015 04:21 PM, Gabriel Ferreira wrote: > The problem is that this particular website uses a flash player > application do broadcast the live images of the urban areas. I'm not > too familiar with Flash Applications. I don't know how to deal with > them using Python. I was wondering if someone could help me solve > this problem. The goal is recording & downloading those videos, from > the mentioned website. And, yes, it has to be an automated program > (no manual work). As I said before you need to look into the RTP protocol. There are utilities for proxying and dumping the stream parameters. Once they are known you can pull them in using rtmpdump. Sometimes the stream parameters can be determined from the web page. get_iplayer, for instance, can create rtp parameters for BBC programs and then stream them with rtmpdump, since the parameters seem to be well defined and consistent. But every streaming site is going to be different. I posted the link to the rtmpdump utilities in my other post. You may be to drive rtmpdump from Python. From rustompmody at gmail.com Mon Feb 2 22:53:45 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 2 Feb 2015 19:53:45 -0800 (PST) Subject: Ghost vulnerability Message-ID: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> How many people (actually machines) out here are vulnerable? http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure shows a python 1-liner to check From rosuav at gmail.com Mon Feb 2 23:38:29 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 15:38:29 +1100 Subject: Ghost vulnerability In-Reply-To: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> Message-ID: On Tue, Feb 3, 2015 at 2:53 PM, Rustom Mody wrote: > How many people (actually machines) out here are vulnerable? > > http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure > > shows a python 1-liner to check Well, I have one internal disk server that's vulnerable. It's not accessible to the world, which is why it's still running Ubuntu 10.10, and it's affected. I'm not too concerned about Huix coming under attack. ChrisA From wxjmfauth at gmail.com Tue Feb 3 02:13:06 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Mon, 2 Feb 2015 23:13:06 -0800 (PST) Subject: Create dictionary based of x items per key from two lists In-Reply-To: References: <0dddee06-233b-436a-be48-3c16e62c1718@googlegroups.com> Message-ID: <56384fa2-e2a7-430d-ab2d-e878bedaa15a@googlegroups.com> Le dimanche 1 f?vrier 2015 20:37:14 UTC+1, Ian a ?crit?: > On Sun, Feb 1, 2015 at 11:49 AM, Mark Lawrence wrote: > > On 01/02/2015 18:14, Grant Edwards wrote: > >> A loop containing 1 line of code will execute in the same abount of > >> time as that loop with 1 line of code and 99 blanks lines. > >> > >> The latter loop is running at 100 times as many lines/second as the > >> former. That's _got_ to be better. > >> > > > > At long last my quest seeking the final entry for the Zen of Python is over. > > I'll be delighted to let you accept the honour of raising an issue on the > > bug tracker to get the "this" module changed to reflect my new found wisdom. > > And while you're at it, please also work on changing the name of the > module to "self" -- "this" is for Java. >>> class C: ... def __init__(carot, x): ... carot.x = 999 ... def p(carot): ... print(carot.x) ... >>> c = C(-1) >>> c.p() 999 >>> "self" is not a keyword. --- This will be very funny: age_of_the_captain = int def GetCommandantAge(i: age_of_the_captain) -> age_of_the_captain: Allowed? age_of_the_captain = int cardinal = int BosonSpin = cardinal # can be zero def GetCommanderAge(i: age_of_the_captain) -> BosonSpin: Propose a clean language, instead of blaming those who are writing bad code. From yacinechaouche at yahoo.com Tue Feb 3 04:06:42 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 01:06:42 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: <54cfc65c$0$2887$e4fe514c@news.xs4all.nl> References: <54cfc65c$0$2887$e4fe514c@news.xs4all.nl> Message-ID: <993a3bad-b98b-46b6-b5cd-7883f89f9a29@googlegroups.com> Hello Irmen, > Why? If you have multiple threads serving some requests at the same time, doesn't that > already achieve your goal? Having multiple requests running at a time is one thing. Making them non-blocking is another. That's how I understand it. > In other words, have you tried what you describe above? Yes I already tried that, it's in the source code I posted if you look closely you'll see that I am inherting from the mixin. The server still hangs sometime, until I send it the SIGINT signal. It then sort of drops the hanging socket and resumes normal operations. > (make sure you close the connection correctly or you'll be hogging a thread which may > eventually make the server non responsive) > > Irmen I didn't see any such code in the tests that came with the standard python library (which I took as a starting point example to write my script). Maybe it's already taken care of by some lower-level internal stuff ? From yacinechaouche at yahoo.com Tue Feb 3 04:08:29 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 01:08:29 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: Message-ID: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> Thank you Amirouch. I was hoping to use something very simple and already provided by the standard library. If I can fix the script, the better. If the script can't be fixed, then I'll switch to another library (I already have one in mind). From jeanpierreda at gmail.com Tue Feb 3 04:24:38 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Tue, 3 Feb 2015 01:24:38 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 6:07 AM, Steven D'Aprano wrote: > Run this code: > > # === cut === > > class K(object): > def f(self): pass > > def f(self): pass > > instance = K() > things = [instance.f, f.__get__(instance, K)] > from random import shuffle > shuffle(things) > print(things) > > # === cut === > > > You allege that one of these things is a method, and the other is not. I > challenge you to find any behavioural or functional difference between the > two. (Object IDs don't count.) > > If you can find any meaningful difference between the two, I will accept > that methods have to be created as functions inside a class body. In this particular case, there is none. What if the body of the "method" was super().f() ? Some methods can be defined outside of the body and still work exactly the same, but others won't. > Otherwise you are reduced to claiming that there is some sort of mystical, > undetectable "essence" or "spirit" that makes one of those two objects a > real method and the other one a fake method, even though they have the same > type, the same behaviour, and there is no test that can tell you which is > which. It isn't mystical. There are differences in semantics of defining methods inside or outside of a class that apply in certain situations (e.g. super(), metaclasses). You have cherrypicked an example that avoids them. If one wants to say "A method can (...) by using super()", then methods must be defined to only exist inside of class bodies. Obviously, once you construct the correct runtime values, behavior might be identical. The difference is in whether you can do different things, not in behavior. >> For an example we can all agree on, this is not an instance of >> collections.Iterable, but the docs claim it is iterable: >> https://docs.python.org/2/glossary.html#term-iterable >> >> class MyIterable(object): >> def __getitem__(self, i): return i > > "Iterable" is a generic term, not a type. Despite the existence of the > collections.Iterable ABC, "iterable" refers to any type which can be > iterated over, using either of two different protocols. > > As I said above, if you wanted to argue that "method" was a general term for > any callable attached to an instance or class, then you might have a point. > But you're doing something much weirder: you are arguing that given two > objects which are *identical* save for their object ID, one might be called > a method, and the other not, due solely to where it was created. Not even > where it was retrieved from, but where it was created. > > If you believe that "method or not" depends on where the function was > defined, then this will really freak you out: > > > py> class Q: > ... def f(self): pass # f defined inside the class > ... > py> def f(self): pass # f defined outside the class > ... > py> f, Q.f = Q.f, f # Swap the "inside" f and the "outside" f. > py> instance = Q() > py> instance.f # Uses "outside" f, so not a real method! > > > py> MethodType(f, instance) # Uses "inside" f, so is a real method! > > You are really missing the point, if you think that surprises me. -- Devin From jeanpierreda at gmail.com Tue Feb 3 04:25:35 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Tue, 3 Feb 2015 01:25:35 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf87b8$0$12991$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf87b8$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 2, 2015 at 6:20 AM, Steven D'Aprano wrote: > Devin Jeanpierre wrote: >> Oops, I just realized why such a claim might be made: the >> documentation probably wants to be able to say that any method can use >> super(). So that's why it claims that it isn't a method unless it's >> defined inside a class body. > > You can use super anywhere, including outside of classes. The only thing you > can't do is use the Python 3 "super hack" which automatically fills in the > arguments to super if you don't supply them. That is compiler magic which > truly does require the function to be defined inside a class body. But you > can use super outside of classes: Obviously, I was referring to no-arg super. Please assume good faith and non-ignorance on my part. -- Devin From marko at pacujo.net Tue Feb 3 04:27:01 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 11:27:01 +0200 Subject: How to write a non blocking SimpleHTTPRequestHandler ? References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> Message-ID: <874mr3ibve.fsf@elektro.pacujo.net> Yassine Chaouche : > I was hoping to use something very simple and already provided by the > standard library. The standard library and nonblocking can't be used in the same sentence. That is, unless and until you go asyncio. Marko From as at sci.fi Tue Feb 3 04:53:02 2015 From: as at sci.fi (Anssi Saari) Date: Tue, 03 Feb 2015 11:53:02 +0200 Subject: Ghost vulnerability References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> Message-ID: Rustom Mody writes: > How many people (actually machines) out here are vulnerable? > > http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure > > shows a python 1-liner to check Does that check actually work for anyone? That code didn't segfalt on my vulnerable Debian system but it did on my router which isn't (since the router doesn't use glibc). Oh and of course I can't comment on stinkexchange since I don't have whatever mana points they require... From marko at pacujo.net Tue Feb 3 05:38:39 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 12:38:39 +0200 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87zj8vgtzk.fsf@elektro.pacujo.net> Devin Jeanpierre : > It isn't mystical. There are differences in semantics of defining > methods inside or outside of a class that apply in certain situations > (e.g. super(), metaclasses). You have cherrypicked an example that > avoids them. It's slightly sad that Python exposes the two-level attribute lookup. It would be more elegant if, conceptually, all methods were retrieved from the object's attribute dict. That way, the class would be simply a mundane optimization trick instead of a metaphysical entity. A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance?s class has an attribute by that name, the search continues with the class attributes. [] Marko From yacinechaouche at yahoo.com Tue Feb 3 05:47:16 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 02:47:16 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: <874mr3ibve.fsf@elektro.pacujo.net> References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: On Tuesday, February 3, 2015 at 10:27:11 AM UTC+1, Marko Rauhamaa wrote: > The standard library and nonblocking can't be used in the same sentence. Thanks Marko. It's a lost cause then. I am thinking about switching to one of the following : - CherryPy - Bottle - circuits - Quixote - Weblayer. If anybody has a pointer to an already made comparison I'm a taker. Thanks for your help ! From rosuav at gmail.com Tue Feb 3 05:49:32 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 21:49:32 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <87zj8vgtzk.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 3, 2015 at 9:38 PM, Marko Rauhamaa wrote: > It's slightly sad that Python exposes the two-level attribute lookup. It > would be more elegant if, conceptually, all methods were retrieved from > the object's attribute dict. That way, the class would be simply a > mundane optimization trick instead of a metaphysical entity. > That's the ECMAScript model of classes - prototype-based. It's not Python's. There are many different ways to do things. ChrisA From rosuav at gmail.com Tue Feb 3 05:54:49 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 21:54:49 +1100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 3, 2015 at 9:47 PM, Yassine Chaouche wrote: > On Tuesday, February 3, 2015 at 10:27:11 AM UTC+1, Marko Rauhamaa wrote: >> The standard library and nonblocking can't be used in the same sentence. > > Thanks Marko. It's a lost cause then. You trimmed out the part where he mentioned asyncio. :) ChrisA From yacinechaouche at yahoo.com Tue Feb 3 06:04:59 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 03:04:59 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: > > Thanks Marko. It's a lost cause then. > > You trimmed out the part where he mentioned asyncio. :) > > ChrisA IIRC asyncio is python 3 only and I'm not ready yet to make the leap. From rosuav at gmail.com Tue Feb 3 06:07:50 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Feb 2015 22:07:50 +1100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 3, 2015 at 10:04 PM, Yassine Chaouche wrote: >> > Thanks Marko. It's a lost cause then. >> >> You trimmed out the part where he mentioned asyncio. :) >> >> ChrisA > > IIRC asyncio is python 3 only and I'm not ready yet to make the leap. Then you're stuck with whatever you have, because the Py2 standard library isn't being expanded any. Why not make the leap? Py3 has a lot of advantages over Py2. ChrisA From steve+comp.lang.python at pearwood.info Tue Feb 3 06:19:20 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 03 Feb 2015 22:19:20 +1100 Subject: Ghost vulnerability References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> Message-ID: <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> Anssi Saari wrote: > Rustom Mody writes: > >> How many people (actually machines) out here are vulnerable? >> >> http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure >> >> shows a python 1-liner to check > > Does that check actually work for anyone? That code didn't segfalt on my > vulnerable Debian system but it did on my router which isn't (since the > router doesn't use glibc). Oh and of course I can't comment on > stinkexchange since I don't have whatever mana points they require... Here's the one-liner: python -c 'import socket;y="0"*50000000;socket.gethostbyname(y)' I think it is likely that y="0"*50000000 would segfault due to lack of memory on many machines. I wouldn't trust this as a test. -- Steven From yacinechaouche at yahoo.com Tue Feb 3 06:23:53 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 03:23:53 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: <2b73e0ba-c50d-4abe-af36-ff586e643450@googlegroups.com> Thanks Chris, it's only a matter of time, I'll eventually make the transition to python3 when I'll learn it well enough. From marko at pacujo.net Tue Feb 3 06:27:52 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 13:27:52 +0200 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> Message-ID: <87vbjjgrpj.fsf@elektro.pacujo.net> Chris Angelico : > On Tue, Feb 3, 2015 at 9:38 PM, Marko Rauhamaa wrote: >> It's slightly sad that Python exposes the two-level attribute lookup. >> It would be more elegant if, conceptually, all methods were retrieved >> from the object's attribute dict. That way, the class would be simply >> a mundane optimization trick instead of a metaphysical entity. > > That's the ECMAScript model of classes - prototype-based. It's not > Python's. There are many different ways to do things. For (almost) all practical purposes, that is the Python way as well. If object instantiation (conceptually) copied the class's methods into the object's dict, you'd get the semantics I'm looking for. Marko From greg.ewing at canterbury.ac.nz Tue Feb 3 06:32:09 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 00:32:09 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > Both K.f and K.g are methods, even though only one meets the definition > given in the glossary. The glossary is wrong. > > Or rather, it is not so much that it is *wrong*, but that it is incomplete > and over-simplified. I agree with that; a more complete definition would be "a function that is found in a class as a result of an attribute lookup on an instance of that class". > I defined a method: > > py> from types import MethodType > py> type(instance.f) is MethodType > True Being of type MethodType is not the defining characterisic of a method. MethodType is actually misnamed; an instance of MethodType is *not* a method, in the same way that an eggcup is not an egg. A better name would be MethodWrapper, but that still doesn't mean that anything you wrap with it is a method. An eggcup *usually* contains an egg, but it could contain something else. > instance.f is a method by the glossary definition. Not by the glossary definition as written, since the function you wrapped in a MethodType was not defined inside a class body. -- Greg From marko at pacujo.net Tue Feb 3 06:35:20 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 13:35:20 +0200 Subject: How to write a non blocking SimpleHTTPRequestHandler ? References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: <87r3u7grd3.fsf@elektro.pacujo.net> Chris Angelico : > On Tue, Feb 3, 2015 at 10:04 PM, Yassine Chaouche >> IIRC asyncio is python 3 only and I'm not ready yet to make the leap. > > Then you're stuck with whatever you have, because the Py2 standard > library isn't being expanded any. Why not make the leap? Py3 has a lot > of advantages over Py2. I'm all for Py3, but I'm not ready to conclude asyncio is the way to go. The coroutines haven't won me over. The programming model is quite messy and simply weird. So far I've been happy with select.epoll(), socket.socket() and ten fingers. Marko From alister.nospam.ware at ntlworld.com Tue Feb 3 06:54:59 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Tue, 03 Feb 2015 11:54:59 GMT Subject: Downloading videos (in flash applications) using python References: <317e4983-7deb-4557-91d3-00c9e3991739@googlegroups.com> <40193f0c-fe96-423b-9cf8-8b0e98530613@googlegroups.com> <87vbjk15id.fsf@jester.gateway.sonic.net> <54cfd799$0$2831$e4fe514c@news2.news.xs4all.nl> <9b2d7ee4-2f2f-4e14-9a2c-7d1531147280@googlegroups.com> Message-ID: On Mon, 02 Feb 2015 15:21:07 -0800, Gabriel Ferreira wrote: > Mark Lawrence wrote: > >> I don't actually know, but could you please provide some context and >> write in plain English, those damn ... things are extremely annoying. >> >> > Hi, Mark. > > I am developing a research project, which includes video analysis > (computer vision, big data, data mining, etc). The first part of the > project is about building a database containing a big amount of video of > urban zones. I live in Brazil, so the plan is collecting videos from > urban areas of Brazilian big cities, like Sao Paulo. > > I have to make this task "automatic". in other words, I don't want to > manually download hundreds or thousands hours of videos. So I have to > develop a "robot" to do that assingment for me. (I wish I could do that > using Python). > > I have found a good website that provides me live images from several > cities in Brazil. So that's it! That's what I need. I was expecting to > develop a program to record & download those videos, that are being > broadcasted live on that website. So I would be able to form my database > and continue my research. > > The problem is that this particular website uses a flash player > application do broadcast the live images of the urban areas. I'm not too > familiar with Flash Applications. I don't know how to deal with them > using Python. I was wondering if someone could help me solve this > problem. The goal is recording & downloading those videos, from the > mentioned website. And, yes, it has to be an automated program (no > manual work). > > I wish I had been clear with you guys! > > THX! So why not speak to the owners of the web site & see if thay can provide you with easier access to their video streams if this is a genuine research project I am sure they would like to help -- If God had a beard, he'd be a UNIX programmer. From greg.ewing at canterbury.ac.nz Tue Feb 3 06:58:49 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 00:58:49 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54cf68eb$0$12996$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54c83ab4$0$12982$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf68eb$0$12996$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > In Python 2, they are methods. In Python 3, they are functions, and aren't > converted into methods until you access them via the instance: They're methods in both cases. They don't have to be "converted into methods"; they already are, by virtue of their location and intended usage. The wrapper that gets created on attribute access isn't the method (despite being called MethodType!) -- the method is the function that it wraps. The wrapper is just part of the machinery that passes the self argument to the method. -- Greg From breamoreboy at yahoo.co.uk Tue Feb 3 07:10:05 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 03 Feb 2015 12:10:05 +0000 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> Message-ID: On 03/02/2015 11:04, Yassine Chaouche wrote: > >>> Thanks Marko. It's a lost cause then. >> >> You trimmed out the part where he mentioned asyncio. :) >> >> ChrisA > > IIRC asyncio is python 3 only and I'm not ready yet to make the leap. > The leap from Python 2 to Python 3 is about as high as the second obstacle is this soon to be Olympic sport https://www.youtube.com/watch?v=TSqkdcT25ss -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From nobody at nowhere.invalid Tue Feb 3 07:17:07 2015 From: nobody at nowhere.invalid (Nobody) Date: Tue, 03 Feb 2015 12:17:07 +0000 Subject: Is there a cairo like surface for the screen without the window hassle References: Message-ID: On Mon, 02 Feb 2015 14:20:56 +0100, Antoon Pardon wrote: > I need to have a program construct a number of designs. Of course I can > directly use a pfd surface and later use a pdf viewer to check. But that > becomes rather cumbersome fast. But if I use a cairo-surface for on the > screen I suddenly have to cope with expose events and all such things I > am not really interested in. If you're writing a GUI application, something has to deal with those events regardless of which rendering API you use. > So does someone know of a package that provides a cairo like surface but > that would take care of the events in a rather straight forward matter, > so that my program could make it's design in a window on the screen just > as if it is designing it in a pdf file. What exactly is the issue here? That you want to be able to construct a graphical representation then discard the data used to construct it, rather than having to keep it around in order to handle subsequent expose events? If that's the case, the simplest solution is probably to render to an image surface then handle expose events by drawing the image onto the screen (or handing the image to some kind of "widget" which does this for you). Or, if you want to support dynamic scaling, you can "render" to an SVG surface instead, but widgets which can display SVG aren't as common as those which deal with raster formats. Another solution is to render to a recording surface. This can then be used as a source surface, so handling expose events boils down to a single cairo_paint() operation with the recording surface as the source surface and the window as the destination surface. Maybe there's even a widget somewhere which does this, but I don't know of one. From yacinechaouche at yahoo.com Tue Feb 3 07:56:17 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 04:56:17 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: <87r3u7grd3.fsf@elektro.pacujo.net> References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: On Tuesday, February 3, 2015 at 12:35:32 PM UTC+1, Marko Rauhamaa wrote: > So far I've been happy with select.epoll(), socket.socket() and ten > fingers. > Marko There's already software written to take care of much of the HTTP stuff protocol stuff, the headers etc. I wouldn't rewrite it. I prefer to monkey patch parts of existing code rather then rewrite all of it. But your comment is interesting because, as I understand it, a non-blocking web server is simply a matter of setting timeouts on sockets, catch the exceptions and move on. I don't know why wouldn't that be possible with python stdlib ? > The standard library and nonblocking can't be used in the same sentence. ? From rosuav at gmail.com Tue Feb 3 08:21:31 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 00:21:31 +1100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 3, 2015 at 11:56 PM, Yassine Chaouche wrote: > But your comment is interesting because, as I understand it, a non-blocking web server is simply a matter of setting timeouts on sockets, catch the exceptions and move on. I don't know why wouldn't that be possible with python stdlib ? > Not really. You could, in theory, set very low timeouts and then poll everything, but it's not efficient. What you want to do is say to the system "Hey, see all these sockets? Let me know when *any one of them* has stuff for me", where "stuff" would be a new connected client if it's a listening socket, or some data written if it's a connected socket; and you might need to check if there's room to write more data, too, which you can do with the same syscall. The key here is that you have a long timeout on the meta-event "any one of these being ready". That's not simply a matter of setting socket timeouts; you need a way to handle the meta-event, and that's something along the lines of select(): http://linux.die.net/man/2/select Other languages have inbuilt asynchronous I/O handlers; eg Pike handles this fairly well, and I've made some use of it with a generic networking system: https://github.com/Rosuav/Hogan Basically, you spin up a server with any number of listening sockets, each of which can talk to any number of connected clients, and all of those sockets get smoothly multiplexed on a single thread. Lots of other languages have similar facilities. Python 2.x doesn't have anything of that nature; Python 3's asyncio is exactly that. ChrisA From marko at pacujo.net Tue Feb 3 08:45:55 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 15:45:55 +0200 Subject: How to write a non blocking SimpleHTTPRequestHandler ? References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: <87iofjglbg.fsf@elektro.pacujo.net> Yassine Chaouche : > On Tuesday, February 3, 2015 at 12:35:32 PM UTC+1, Marko Rauhamaa wrote: >> So far I've been happy with select.epoll(), socket.socket() and ten >> fingers. > > [...] > > But your comment is interesting because, as I understand it, a > non-blocking web server is simply a matter of setting timeouts on > sockets, catch the exceptions and move on. Now I think you might have some misconceptions about nonblocking networking I/O. Nonblocking I/O is done using asynchronous, or event-driven, programming. Your code reacts to external stimuli, never blocking, mostly just sleeping. The reactions are defined in callback routings, aka listeners, aka event handlers. > I don't know why wouldn't that be possible with python stdlib ? It is possible using the low-level facilities. However, the traditional high-level facilities are built on multithreading, which (as a rule) is based on blocking I/O. Marko From rosuav at gmail.com Tue Feb 3 09:03:37 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 01:03:37 +1100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: <87iofjglbg.fsf@elektro.pacujo.net> References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> <87iofjglbg.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 4, 2015 at 12:45 AM, Marko Rauhamaa wrote: >> But your comment is interesting because, as I understand it, a >> non-blocking web server is simply a matter of setting timeouts on >> sockets, catch the exceptions and move on. > > Now I think you might have some misconceptions about nonblocking > networking I/O. Nonblocking I/O is done using asynchronous, or > event-driven, programming. Your code reacts to external stimuli, never > blocking, mostly just sleeping. The reactions are defined in callback > routings, aka listeners, aka event handlers. Not strictly true - that's just one convenient way of doing things. A callback/event-handler structure lets you write a bunch of listeners that coexist effortlessly, but it's not the only way to do non-blocking I/O, and it's certainly not an intrinsic part of the concept. That said, though, it is a VERY convenient way to lay things out in the code. The Pike system I offered, and most of the older multiplexed-I/O systems I've used, did work that way. It just isn't something that non-blocking necessarily implies. >> I don't know why wouldn't that be possible with python stdlib ? > > It is possible using the low-level facilities. However, the traditional > high-level facilities are built on multithreading, which (as a rule) is > based on blocking I/O. Multithreading is another way to cope with the same problem of wanting to deal with different sockets on a single CPU, but I don't think it's inherently a part of any of Python's own high-level facilities - not that I can think of, at least? However, in terms of common programming models, yes, multithreading+blocking I/O is an effective way to write code, and will therefore be commonly used. I wish more people had grown up on OS/2 instead of (or as well as) Windows or Unix. Threading is not such a bugbear as a lot of people seem to think. Yes, some platforms have traditionally had poor implementations, and to be sure, you don't want to mix threading and forking without a *lot* of care, but threads aren't inherently bad. They're a useful tool in the toolbox. Sometimes non-blocking I/O is the right thing to do; sometimes threads suit the problem better; other times, something else again. ChrisA From ian.g.kelly at gmail.com Tue Feb 3 09:07:12 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 3 Feb 2015 07:07:12 -0700 Subject: __pycache__ In-Reply-To: References: Message-ID: On Mon, Feb 2, 2015 at 4:26 PM, Luke Lloyd wrote: > I am in school and there is a problem with my code: > > > > When I try to run my code in the python code in the python shell it waits > about 10 seconds then shows an error that says ?IDLE?s subprocess didn?t > make connection. Either IDLE can?t start a subprocess or personal firewall > software is blocking the connection.? Then when I press OK it just closes. IDLE runs code by starting a Python interpreter in a subprocess and connecting to it. If this is a school computer then I would guess it probably has a very aggressive firewall as the message suggests. What happens if you try running the script directly from the OS command line instead of through IDLE? From ian.g.kelly at gmail.com Tue Feb 3 09:09:34 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 3 Feb 2015 07:09:34 -0700 Subject: __pycache__ In-Reply-To: References: Message-ID: On Mon, Feb 2, 2015 at 4:26 PM, Luke Lloyd wrote: > I am in school and there is a problem with my code: > > > > When I try to run my code in the python code in the python shell it waits > about 10 seconds then shows an error that says ?IDLE?s subprocess didn?t > make connection. Either IDLE can?t start a subprocess or personal firewall > software is blocking the connection.? Then when I press OK it just closes. > > > > P.S: My code is encrypting and decrypting using an offset factor. By the way, your subject line doesn't seem to have anything to do with your question. From amirouche.boubekki at gmail.com Tue Feb 3 09:09:51 2015 From: amirouche.boubekki at gmail.com (Amirouche Boubekki) Date: Tue, 03 Feb 2015 14:09:51 +0000 Subject: How to write a non blocking SimpleHTTPRequestHandler ? References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: > The standard library and nonblocking can't be used in the same sentence. python 2.x stdlib has no high level support of *async* code. There is trollius library that ports asyncio to py2 though. I was a bit quick in my first anwser. What you want is to prevent the socket to wait indefinetly for data (based on strace output) which is done with socket.setblocking/settimeout [1]. asynchronous (asyncio) is something else, and you would still need to handle blocking I think. There is contentbrowser [2] which is somekind of web proxy. IMO, python 2 -> python 3 is not a big leap. Some things are better in python 3. [1] https://docs.python.org/2/library/socket.html#socket.socket.setblocking [2] https://bitbucket.org/david/contentbrowser/src On Tue Feb 03 2015 at 2:00:27 PM Yassine Chaouche wrote: > On Tuesday, February 3, 2015 at 12:35:32 PM UTC+1, Marko Rauhamaa wrote: > > > So far I've been happy with select.epoll(), socket.socket() and ten > > fingers. > > Marko > > There's already software written to take care of much of the HTTP stuff > protocol stuff, the headers etc. I wouldn't rewrite it. I prefer to monkey > patch parts of existing code rather then rewrite all of it. > > But your comment is interesting because, as I understand it, a > non-blocking web server is simply a matter of setting timeouts on sockets, > catch the exceptions and move on. I don't know why wouldn't that be > possible with python stdlib ? > > > > The standard library and nonblocking can't be used in the same sentence. > > ? > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marko at pacujo.net Tue Feb 3 09:20:40 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 16:20:40 +0200 Subject: How to write a non blocking SimpleHTTPRequestHandler ? References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> <87iofjglbg.fsf@elektro.pacujo.net> Message-ID: <87a90vgjpj.fsf@elektro.pacujo.net> Chris Angelico : > Threading is not such a bugbear as a lot of people > seem to think. Yes, some platforms have traditionally had poor > implementations Java has excellent multithreading facilities. Still, I have seen seasoned Java developers commit atrocious crimes against thread-safety that are impossible to troubleshoot and repair afterwards. There are ways to manage the complications of multithreading, but no universal practices are agreed upon so you can't count on other/legacy software to obey your policies or your software obey theirs. And even under the best guidelines there are seemingly intractable cornercases where you feel helplessly lost. Marko From emile at fenx.com Tue Feb 3 09:34:27 2015 From: emile at fenx.com (Emile van Sebille) Date: Tue, 03 Feb 2015 06:34:27 -0800 Subject: __pycache__ In-Reply-To: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> References: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> Message-ID: On 2/3/2015 6:21 AM, Dennis Lee Bieber wrote: > The second is to use Google... > > https://www.google.com/?gws_rd=ssl#q=python+idle+can%27t+make+connection > > but the first page of results isn't helping -- lots of reports of the > problem, but no firm remedy listed. it was suggested to me recently that i include 'solved' in google searches. doesn't always work, but helps when i don't find a result quickly. From rosuav at gmail.com Tue Feb 3 09:39:42 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 01:39:42 +1100 Subject: __pycache__ In-Reply-To: References: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> Message-ID: On Wed, Feb 4, 2015 at 1:34 AM, Emile van Sebille wrote: > On 2/3/2015 6:21 AM, Dennis Lee Bieber wrote: > >> The second is to use Google... >> >> https://www.google.com/?gws_rd=ssl#q=python+idle+can%27t+make+connection >> >> but the first page of results isn't helping -- lots of reports of the >> problem, but no firm remedy listed. > > > it was suggested to me recently that i include 'solved' in google searches. > doesn't always work, but helps when i don't find a result quickly. I'd not do that initially, as it'll cause a lot of false negatives; but if you're finding a bunch of problems and no solutions, then yes, that could help. ChrisA From yacinechaouche at yahoo.com Tue Feb 3 09:50:11 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Tue, 3 Feb 2015 06:50:11 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: <3b558129-df61-4a3d-9cca-17e5340c33a3@googlegroups.com> On Tuesday, February 3, 2015 at 3:17:37 PM UTC+1, Amirouche Boubekki wrote: > What you want is to prevent the socket to wait indefinetly for data (based on strace output) which is done with socket.setblocking/settimeout [1] Exactly ! thanks for taking time to reading my original post a second time. Maybe I didn't express my problem well enough and the discussion has maybe drifted to an XYProblem type discussion :) I was about to say that most webframeworks for python already take care of the problem I'm facing, so I certainly confused people with improper formulation. > There is contentbrowser [2] which is somekind of web proxy. It's using werkzeug and it seems to be a full web application. I just want something that is right above sockets and selects. All that my script is doing is grabbing some json data from an external url provided by the browser and return a formatted page showing the json data in human-readable HTML pages. Nothing fancy. From philosganga at gmail.com Tue Feb 3 10:03:47 2015 From: philosganga at gmail.com (Filadelfo Fiamma) Date: Tue, 3 Feb 2015 16:03:47 +0100 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: <3b558129-df61-4a3d-9cca-17e5340c33a3@googlegroups.com> References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> <3b558129-df61-4a3d-9cca-17e5340c33a3@googlegroups.com> Message-ID: I think You can try the asyncore lib: https://docs.python.org/2/library/asyncore.html 2015-02-03 15:50 GMT+01:00 Yassine Chaouche < yacinechaouche at yahoo.com.dmarc.invalid>: > On Tuesday, February 3, 2015 at 3:17:37 PM UTC+1, Amirouche Boubekki wrote: > > What you want is to prevent the socket to wait indefinetly for data > (based on strace output) which is done with socket.setblocking/settimeout > [1] > > Exactly ! thanks for taking time to reading my original post a second > time. Maybe I didn't express my problem well enough and the discussion has > maybe drifted to an XYProblem type discussion :) > > I was about to say that most webframeworks for python already take care of > the problem I'm facing, so I certainly confused people with improper > formulation. > > > There is contentbrowser [2] which is somekind of web proxy. > > It's using werkzeug and it seems to be a full web application. I just want > something that is right above sockets and selects. All that my script is > doing is grabbing some json data from an external url provided by the > browser and return a formatted page showing the json data in human-readable > HTML pages. Nothing fancy. > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Filadelfo Fiamma mail: philosganga at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoon.pardon at rece.vub.ac.be Tue Feb 3 10:46:29 2015 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 03 Feb 2015 16:46:29 +0100 Subject: Is there a cairo like surface for the screen without the window hassle In-Reply-To: References: Message-ID: <54D0ED55.8030107@rece.vub.ac.be> Op 03-02-15 om 13:17 schreef Nobody: > On Mon, 02 Feb 2015 14:20:56 +0100, Antoon Pardon wrote: > >> I need to have a program construct a number of designs. Of course I can >> directly use a pfd surface and later use a pdf viewer to check. But that >> becomes rather cumbersome fast. But if I use a cairo-surface for on the >> screen I suddenly have to cope with expose events and all such things I >> am not really interested in. > > If you're writing a GUI application, something has to deal with those > events regardless of which rendering API you use. Sure but from my point of view I am not writing a GUI. I just have a function that takes a cairo context and draws an image on it. That works all very fine if it is a postscript of pdf or an image but gives trouble when the context is associated with a window on the screen. >> So does someone know of a package that provides a cairo like surface but >> that would take care of the events in a rather straight forward matter, >> so that my program could make it's design in a window on the screen just >> as if it is designing it in a pdf file. > > What exactly is the issue here? That you want to be able to construct a > graphical representation then discard the data used to construct it, > rather than having to keep it around in order to handle subsequent expose > events? I want those expose events handled for me, so I don't have to care for them, so that I can just use a context that will render the drawing in a window on the screen without me having to wonder about any event. > If that's the case, the simplest solution is probably to render to an > image surface then handle expose events by drawing the image onto the > screen (or handing the image to some kind of "widget" which does this for > you). > > Another solution is to render to a recording surface. This can then be > used as a source surface, so handling expose events boils down to a single > cairo_paint() operation with the recording surface as the source surface > and the window as the destination surface. Maybe there's even a widget > somewhere which does this, but I don't know of one. Yes I had written something like this for pygtk. The module provided an initiator which you called with a function. This would then start the gtk.main loop and start the function in a thread. The module provided also a class similar to a cairo context, which would create a windon and draw into it, it was implemented more or less as you describe above. The main advantage was that the function that made the design didn't need to care about being used in a GUI. Problem is I don't succeed in getting it to work with the new introspective gtk framework. But since this seems something usefull I thought I ask about maybe someone else already having written something similar before I sink more of my own time into it. From ginobusciarello at outlook.com Tue Feb 3 11:07:00 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Tue, 3 Feb 2015 08:07:00 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From breamoreboy at yahoo.co.uk Tue Feb 3 11:31:06 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 03 Feb 2015 16:31:06 +0000 Subject: __pycache__ In-Reply-To: References: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> Message-ID: On 03/02/2015 14:34, Emile van Sebille wrote: > On 2/3/2015 6:21 AM, Dennis Lee Bieber wrote: > >> The second is to use Google... >> >> >> https://www.google.com/?gws_rd=ssl#q=python+idle+can%27t+make+connection >> >> but the first page of results isn't helping -- lots of reports of the >> problem, but no firm remedy listed. > > it was suggested to me recently that i include 'solved' in google > searches. doesn't always work, but helps when i don't find a result > quickly. > I know that you can target sites, but can you exclude them, e.g. -site:stackoverflow.com ? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From mimmocubiddu at yahoo.com Tue Feb 3 11:36:23 2015 From: mimmocubiddu at yahoo.com (MICHELE RAGAZZI. ODEY GIANO.) Date: Tue, 3 Feb 2015 08:36:23 -0800 (PST) Subject: -- redacted -- In-Reply-To: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> References: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> Message-ID: <04a9e910-6eb1-4181-998e-e01a3ea86a3a@googlegroups.com> -- redacted -- From mimmocubiddu at yahoo.com Tue Feb 3 11:37:05 2015 From: mimmocubiddu at yahoo.com (MICHELE RAGAZZI. ODEY GIANO.) Date: Tue, 3 Feb 2015 08:37:05 -0800 (PST) Subject: -- redacted -- In-Reply-To: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: <41457f76-c752-48a5-999b-3f7acc593924@googlegroups.com> -- redacted -- From breamoreboy at yahoo.co.uk Tue Feb 3 11:39:02 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 03 Feb 2015 16:39:02 +0000 Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> <3b558129-df61-4a3d-9cca-17e5340c33a3@googlegroups.com> Message-ID: On 03/02/2015 15:03, Filadelfo Fiamma wrote: > I think You can try the asyncore lib: > https://docs.python.org/2/library/asyncore.html > People can try it but it's effectively deprecated, with its partner asynchat, in favour of asyncio. Also please don't top post here, thank you. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From emile at fenx.com Tue Feb 3 11:44:52 2015 From: emile at fenx.com (Emile van Sebille) Date: Tue, 03 Feb 2015 08:44:52 -0800 Subject: __pycache__ In-Reply-To: References: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> Message-ID: On 2/3/2015 8:31 AM, Mark Lawrence wrote: > On 03/02/2015 14:34, Emile van Sebille wrote: >> On 2/3/2015 6:21 AM, Dennis Lee Bieber wrote: >> >>> The second is to use Google... >>> >>> >>> https://www.google.com/?gws_rd=ssl#q=python+idle+can%27t+make+connection >>> >>> but the first page of results isn't helping -- lots of reports of the >>> problem, but no firm remedy listed. >> >> it was suggested to me recently that i include 'solved' in google >> searches. doesn't always work, but helps when i don't find a result >> quickly. >> > > I know that you can target sites, but can you exclude them, e.g. > -site:stackoverflow.com ? > yep -- https://support.google.com/websearch/answer/2466433?hl=en - Remove words When you use a dash before a word or site, it excludes results that include that word or site. This is useful for words with multiple meanings, like Jaguar the car brand and jaguar the animal. Examples: jaguar speed -car and pandas -site:wikipedia.org From skip.montanaro at gmail.com Tue Feb 3 11:47:23 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 3 Feb 2015 10:47:23 -0600 Subject: __pycache__ In-Reply-To: References: <4ml1da9lmjhjemfljt29f4ts9p75c499tl@4ax.com> Message-ID: On Tue, Feb 3, 2015 at 10:31 AM, Mark Lawrence wrote: > I know that you can target sites, but can you exclude them, e.g. > -site:stackoverflow.com ? Yes, you can. Compare the results of these two searches, for example: git initial push git initial push -site:stackoverflow.com Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From torriem at gmail.com Tue Feb 3 12:31:54 2015 From: torriem at gmail.com (Michael Torrie) Date: Tue, 03 Feb 2015 10:31:54 -0700 Subject: Ghost vulnerability In-Reply-To: <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54D1060A.30306@gmail.com> On 02/03/2015 04:19 AM, Steven D'Aprano wrote: > Anssi Saari wrote: > >> Rustom Mody writes: >> >>> How many people (actually machines) out here are vulnerable? >>> >>> > http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure >>> >>> shows a python 1-liner to check >> >> Does that check actually work for anyone? That code didn't segfalt on my >> vulnerable Debian system but it did on my router which isn't (since the >> router doesn't use glibc). Oh and of course I can't comment on >> stinkexchange since I don't have whatever mana points they require... > > Here's the one-liner: > > python -c 'import socket;y="0"*50000000;socket.gethostbyname(y)' > > > I think it is likely that y="0"*50000000 would segfault due to lack of > memory on many machines. I wouldn't trust this as a test. I ran it on both my servers (each running a different version of the OS) which were recently updated to Red Hat's latest version of glibc that fixes the problem, and both of them segfault with this one liner. From steve+comp.lang.python at pearwood.info Tue Feb 3 12:40:22 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 04 Feb 2015 04:40:22 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > On Mon, Feb 2, 2015 at 6:07 AM, Steven D'Aprano > wrote: >> Run this code: >> >> # === cut === >> >> class K(object): >> def f(self): pass >> >> def f(self): pass >> >> instance = K() >> things = [instance.f, f.__get__(instance, K)] >> from random import shuffle >> shuffle(things) >> print(things) >> >> # === cut === >> >> >> You allege that one of these things is a method, and the other is not. I >> challenge you to find any behavioural or functional difference between >> the two. (Object IDs don't count.) >> >> If you can find any meaningful difference between the two, I will accept >> that methods have to be created as functions inside a class body. > > In this particular case, there is none. What if the body of the > "method" was super().f() ? What of it? The zero-argument form of super() is a compiler hack documented as only working inside classes. Most methods don't even use super *at all*, let alone the Python3-only zero argument form. Surely you cannot be serious that the defining characteristic of what is or is not a method is whether or not an explicit hack works? But you know, I can duplicate the compiler hack and still use the zero-argument form of super in a method defined on the outside of the class. py> class A(object): ... pass ... py> def magic(): # Don't do this at home! ... __class__ = A ... def f(self): ... __class__ ... return super() # Magic zero-argument form. ... return f ... py> A.f = magic() py> a = A() py> a.f() , > So to answer your question: What if the body of the "method" was super().f() ? It would still work correctly. Because Python is just that awesome. > Some methods can be defined outside of the body and still work exactly > the same, but others won't. "Some methods can be defined outside of the body". AT LONG LAST THE LIGHT DAWNS! THE PENNY DROPS! If some methods can be defined outside of the body of a class, then being defined inside the body of a class does not define a method. You say "some methods", but the reality is that *all methods* can do so. Even if they use super. Even if they use the zero-argument form of super (although that one is rather inconvenient). The class statement is syntactic sugar for calling type(name, bases, ns): class A(bases): x = 100 f = lambda self: self.x + 1 is equivalent to: A = type("A", bases, {'x': 100, 'f': lambda self: self.x + 1}) *ANY* class you create with the class statement can be created (less conveniently) with type (or the appropriate metaclass). Obviously the class statement form is convenient because it allows you to define the items of the namespace in place, which you can't always do using type itself. With type, you may have to prepare the items first, insert them into a dict, and pass it as the namespace argument. It also offers convenient syntax for changing the metaclass. Most importantly, it is simply more readable and nicer to be able to define the methods indented inside the class statement. This is the natural way to define classes, and given that the glossary need not be 100% complete and definitive, "function defined inside a class body" is close enough to the truth. But it is not *the whole truth*. Not only can methods be defined outside of a class, but with a little preparation functions defined inside of classes can remain functions. py> class function(object): ... def __init__(self, func): ... self.func = func ... def __get__(self, obj, cls=None): ... return self.func ... py> class B(object): ... @function ... def f(x, y): # No self. ... return x + y ... py> B.f(23, 42) 65 py> b = B() py> b.f(23, 42) 65 py> type(b.f) (Alternatively, I could change the metaclass, although that's trickier. Using a custom descriptor is just too easy.) >> Otherwise you are reduced to claiming that there is some sort of >> mystical, undetectable "essence" or "spirit" that makes one of those two >> objects a real method and the other one a fake method, even though they >> have the same type, the same behaviour, and there is no test that can >> tell you which is which. > > It isn't mystical. There are differences in semantics of defining > methods inside or outside of a class that apply in certain situations > (e.g. super(), metaclasses). You have cherrypicked an example that > avoids them. Out of the millions, billions of methods people write, how many do you think use super? 1% 0.1%? 10%? And you accuse *me* of cherrypicking. Not that it matters. As I have shown, even the zero argument form of super can be used. -- Steven From glicerinu at gmail.com Tue Feb 3 12:47:02 2015 From: glicerinu at gmail.com (Marc Aymerich) Date: Tue, 3 Feb 2015 18:47:02 +0100 Subject: Ghost vulnerability In-Reply-To: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> Message-ID: On Tue, Feb 3, 2015 at 4:53 AM, Rustom Mody wrote: > How many people (actually machines) out here are vulnerable? > > > http://security.stackexchange.com/questions/80210/ghost-bug-is-there-a-simple-way-to-test-if-my-system-is-secure > > shows a python 1-liner to check > -- > https://mail.python.org/mailman/listinfo/python-list > Not very reliable in my experience, this python test does segfault on my *patched* Debian machine root at web:~# python -c 'import socket;y="0"*50000000;socket.gethostbyname(y)' Segmentation fault However, the other test proposed on stackechange correctly reports that I'm not vulnerable ;) root at web:/tmp# wget https://webshare.uchicago.edu/orgs/ITServices/itsec/Downloads/GHOST.c root at web:/tmp# gcc GHOST.c -o GHOST root at web:/tmp# ./GHOST not vulnerable -- Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From travisgriggs at gmail.com Tue Feb 3 13:07:46 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Tue, 3 Feb 2015 10:07:46 -0800 Subject: Is there a cairo like surface for the screen without the window hassle In-Reply-To: <54CF79B8.10904@rece.vub.ac.be> References: <54CF79B8.10904@rece.vub.ac.be> Message-ID: <03AA73AE-66A4-4EBF-9518-02D740005AFE@gmail.com> > On Feb 2, 2015, at 5:20 AM, Antoon Pardon wrote: > > I need to have a program construct a number of designs. Of course I can directly > use a pfd surface and later use a pdf viewer to check. But that becomes rather > cumbersome fast. But if I use a cairo-surface for on the screen I suddenly have > to cope with expose events and all such things I am not really interested in. > > So does someone know of a package that provides a cairo like surface but that > would take care of the events in a rather straight forward matter, so that my > program could make it's design in a window on the screen just as if it is > designing it in a pdf file. > For the most part, you cannot draw directly to the screen with Cairo. Some OSes kind of allow out, but they won?t repaint it for you. Any viewing software that will auto detect file updates and reload would do the trick. For example, I know that preview on OS X will automatically reload a png file that I write. Cairo can generate png output. Just open preview on the file, and then have the program rewrite the same file. From as at sci.fi Tue Feb 3 14:38:29 2015 From: as at sci.fi (Anssi Saari) Date: Tue, 03 Feb 2015 21:38:29 +0200 Subject: Ghost vulnerability References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano writes: > Here's the one-liner: > > python -c 'import socket;y="0"*50000000;socket.gethostbyname(y)' > > > I think it is likely that y="0"*50000000 would segfault due to lack of > memory on many machines. I wouldn't trust this as a test. Hmm, how much RAM does that one-liner actually need? My router has 128 MB total RAM with about 90 MB free. So it can store the string once but if it's copied with the gethostbyname call then it'll run out... According to a Reddit thread (http://www.reddit.com/r/Python/comments/2u7ghu/python_socketgethostbyname_is_not_affected_by/) Python's socket.gethostbyname() doesn't actually even call the gethostbyname function in glibc, it uses the newer getaddrinfo instead. So it's a little unlikely to cause a segfault because of the Ghost vuln :) Anyways, here's an example calling gethostbyname directly in python: from ctypes import CDLL o = CDLL('libc.so.6') for i in range(0, 2500): o.gethostbyname('0'*i) I don't have a vulnerable system to test on any more though. From davea at davea.name Tue Feb 3 16:00:00 2015 From: davea at davea.name (Dave Angel) Date: Tue, 03 Feb 2015 16:00:00 -0500 Subject: Benchmarking some modules - strange result In-Reply-To: References: Message-ID: <54D136D0.80903@davea.name> On 01/24/2015 09:36 PM, Dan Stromberg wrote: > On Sat, Jan 24, 2015 at 6:24 PM, Chris Angelico wrote: >> On Sun, Jan 25, 2015 at 1:11 PM, Dan Stromberg wrote: >>> For simplicity, let's say I've been running the suite of performance >>> tests within a single interpreter - so I test one module thoroughly, >>> then move on to the next without exiting the interpreter. >>> [chomp more details] >> >> Do the modules import stuff that's staying imported? That could >> majorly affect performance. What about .pyc files? > > The modules are all imported up at the top of the script. I haven't > verified this, but I'm guessing the .pyc's are created at the > beginning too. > >> I don't know if you can truly reset the interpreter. Snapshotting >> sys.modules and reinstating it afterwards would help, but there are >> other concerns too. I would personally just restart the interpreter >> each time, frankly; it's the easiest way to be sure. > > Maybe so ^_^ > I've been offline for two weeks, so sorry about the delay in responding. There are many pieces of the Python system that do not reset. But just because the C memory allocation routines do not reset to their initial state, running the gc between runs is not sufficient to normalize things. I believe you MUST restart the interpreter between runs. Further, the OS buffer system and the paging system are not going to be in the same state between runs. years ago I knew how to clear those buffers in Windows, but I've never needed to look it up for Linux. in Windows, I'd look first at one of the sysinternalstools, such as https://technet.microsoft.com/en-us/sysinternals/ff700229 On Linux, i'd look at the following utilities, swapon/swapoff, and at file /proc/sys/vm/drop_caches and maybe see website: https://code.google.com/p/linux-ftools/ -- DaveA From priisdk at gmail.com Tue Feb 3 16:00:57 2015 From: priisdk at gmail.com (Poul Riis) Date: Tue, 3 Feb 2015 13:00:57 -0800 (PST) Subject: Cairo module Message-ID: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> I just tried the Cairo Python module. I ran the test file below. It works perfectly but instead of saving the resulting image as a file I want to see it displayed directly on the screen. How can I do that? Poul Riis import math import cairo WIDTH, HEIGHT = 256, 256 surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, WIDTH, HEIGHT) ctx = cairo.Context (surface) ctx.scale (WIDTH, HEIGHT) # Normalizing the canvas pat = cairo.LinearGradient (0.0, 0.0, 0.0, 1.0) pat.add_color_stop_rgba (1, 0.7, 0, 0, 0.5) # First stop, 50% opacity pat.add_color_stop_rgba (0, 0.9, 0.7, 0.2, 1) # Last stop, 100% opacity ctx.rectangle (0, 0, 1, 1) # Rectangle(x0, y0, x1, y1) ctx.set_source (pat) ctx.fill () ctx.translate (0.1, 0.1) # Changing the current transformation matrix ctx.move_to (0, 0) ctx.arc (0.2, 0.1, 0.1, -math.pi/2, 0) # Arc(cx, cy, radius, start_angle, stop_angle) ctx.line_to (0.5, 0.1) # Line to (x,y) ctx.curve_to (0.5, 0.2, 0.5, 0.4, 0.2, 0.8) # Curve(x1, y1, x2, y2, x3, y3) ctx.close_path () ctx.set_source_rgb (0.3, 0.2, 0.5) # Solid color ctx.set_line_width (0.02) ctx.stroke () surface.write_to_png ("example.png") # Output to PNG From greg.ewing at canterbury.ac.nz Tue Feb 3 16:12:59 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 10:12:59 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <87vbjjgrpj.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > For (almost) all practical purposes, that is the Python way as well. If > object instantiation (conceptually) copied the class's methods into the > object's dict, you'd get the semantics I'm looking for. If things worked the way you want, it would be impossible to store a function in an instance attribute and get it out again *without* it being treated as a method and getting 'self' added to its arguments. That would be a considerable nuisance when dealing with callbacks and the like. -- Greg From davea at davea.name Tue Feb 3 16:16:02 2015 From: davea at davea.name (Dave Angel) Date: Tue, 03 Feb 2015 16:16:02 -0500 Subject: Idiomatic backtracking in Python In-Reply-To: References: <87bnlml44b.fsf@elektro.pacujo.net> Message-ID: <54D13A92.5060701@davea.name> On 01/25/2015 08:45 PM, Chris Angelico wrote: > On Mon, Jan 26, 2015 at 12:31 PM, Marko Rauhamaa wrote: >> Backtracking means the part of depth-first traversal where you retreat >> to the parent node. If you implement your traversal with a recursive >> function, backtracking means ? more or less ? a return from the >> function. > > But possibly you need to return from N levels, not just one. Imagine > you're traversing a directory tree using a simplistic recursive > algorithm: > > def traverse(dir): > for d in subdirs(dir): traverse(d) > for f in files(dir): > if file_size(f) > 1024*1024*1024: skip this entire branch of the tree > > Obviously you have to define the branch somehow, but there are plenty > of times when you might want to break out of "everything up to here". > How do you define that? How do you return lots of levels all at once? > I remember facing this exact problem in trying to solve a particular > piece-layout puzzle; if I discovered an impossible situation, I could > actually discard at least two or three levels of recursion all at > once, because there's no way that the situation could become > un-impossible within those levels. Can't remember how I ended up > dealing with that... I think I got naughty and used a global variable. > When I've done things like that, there was no need to do a "return multiple". You just return, and your caller happens to be a the end of his loop, so he returns also. Classic example of this is the 8 queens puzzle. Each level is going to examine one row, and once there are no places that aren't yet attacked, it simply returns. -- DaveA From marko at pacujo.net Tue Feb 3 16:28:54 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 03 Feb 2015 23:28:54 +0200 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> Message-ID: <87egq6adm1.fsf@elektro.pacujo.net> Gregory Ewing : > Marko Rauhamaa wrote: >> For (almost) all practical purposes, that is the Python way as well. If >> object instantiation (conceptually) copied the class's methods into the >> object's dict, you'd get the semantics I'm looking for. > > If things worked the way you want, it would be impossible to store a > function in an instance attribute and get it out again *without* it > being treated as a method and getting 'self' added to its arguments. > That would be a considerable nuisance when dealing with callbacks and > the like. Sorry, you'll have to elaborate. I don't quite follow you. Right now Python generates the trampoline from the class prototype every time you call a method. If the semantics allowed, you could create the trampoline at instantiation time (for better or worse). That way, the problem you seem to be referring to wouldn't materialize. Marko From greg.ewing at canterbury.ac.nz Tue Feb 3 16:39:54 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 10:39:54 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > If some methods can be defined outside of the body of a class, then being > defined inside the body of a class does not define a method. Nobody's disputing that. The business about super() is just a possible reason for the glossary to define the word 'method' in a more restricted way -- because it simplifies the wording of *other* parts of the docs that talk about super(). Another thing to consider is that while tricks like manually inserting __class__ into a function may work today with CPython, they might not work in future versions or other implementations. So there are good reasons for the docs to be conservative about what they promise. Also, with Python being so dynamic, just about *anything* you can say about its usual behaviour can be circumvented with enough hackery. If the docs were to pedantically take all of those possibilities into account at every turn, they would be so dense and impenetrable as to be nearly useless to anyone other than language lawyers. All this started when I pointed out that *if* you take the glossary definition of the term 'method' at its word, then what the docs say about the __dir__ method won't lead you to think that attaching it to an instance would work. That's true regardless of whether you think the glossary definition is too restrictive or not. I wouldn't have thought that this obvservation would be so controversial. But maybe I'm wrong, and Python really is doomed -- to death by language lawyering!-) -- Greg From tjreedy at udel.edu Tue Feb 3 17:01:34 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 03 Feb 2015 17:01:34 -0500 Subject: __pycache__ In-Reply-To: References: Message-ID: On 2/2/2015 6:26 PM, Luke Lloyd wrote: > I am in school and there is a problem with my code: > > When I try to run my code in the python code in the python shell it > waits about 10 seconds then shows an error that says ?IDLE?s subprocess > didn?t make connection. Either IDLE can?t start a subprocess or personal > firewall software is blocking the connection.? Then when I press OK it > just closes. Idle is intended for developing code, not running code that you have already developed. (Though there are cases, at least on Windows, where it is better for running programs than python in a console.) In any case, a solution, given is the Idle doc section, is to start Idle with the -n option. python -m idlelib -n add '-r x.y' to run an existing python file (this is missing from the doc section and needs to be added). -- Terry Jan Reedy From ian.g.kelly at gmail.com Tue Feb 3 17:04:30 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 3 Feb 2015 15:04:30 -0700 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 3, 2015 at 10:40 AM, Steven D'Aprano wrote: > AT LONG LAST THE LIGHT DAWNS! THE PENNY DROPS! Careful there, you're starting to sound like Ranting Rick. ;-) From rosuav at gmail.com Tue Feb 3 17:08:22 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 09:08:22 +1100 Subject: Ghost vulnerability In-Reply-To: References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 4, 2015 at 6:38 AM, Anssi Saari wrote: > Anyways, here's an example calling gethostbyname directly in python: > > from ctypes import CDLL > o = CDLL('libc.so.6') > for i in range(0, 2500): > o.gethostbyname('0'*i) > > I don't have a vulnerable system to test on any more though. That bombs on my internal disk server, which is said to be vulnerable using the C implementation. ChrisA From travisgriggs at gmail.com Tue Feb 3 17:19:30 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Tue, 3 Feb 2015 14:19:30 -0800 Subject: Cairo module In-Reply-To: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> References: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> Message-ID: > On Feb 3, 2015, at 1:00 PM, Poul Riis wrote: > > I just tried the Cairo Python module. > I ran the test file below. > It works perfectly but instead of saving the resulting image as a file I want to see it displayed directly on the screen. > How can I do that? > I have quiet a bit of experience with Cairo (I wrote language bindings for it in Smalltalk and had the time of my life with it there); I have no experience with the pycairo bindings. > > import math > import cairo > > WIDTH, HEIGHT = 256, 256 > > surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, WIDTH, HEIGHT) This is your basic problem right here. And ImageSurface is for creating an Image (sometimes called a raster graphic or bitmap). If you want to display directly to your screen, you need to create a surface that binds to your screen?s display functionality. There is one for each main operating system: Win32Surface XLibSurface QuartzSurface (I see that this is missing from the pycairo documentation, but it is in the cairo documentation, and the pycairo.c file at least has some reference to it) Allocating one of these involves getting handles (and other information) for a given window on screen of your host OS and creating the surface from it. > ctx = cairo.Context (surface) > > ctx.scale (WIDTH, HEIGHT) # Normalizing the canvas > > pat = cairo.LinearGradient (0.0, 0.0, 0.0, 1.0) > pat.add_color_stop_rgba (1, 0.7, 0, 0, 0.5) # First stop, 50% opacity > pat.add_color_stop_rgba (0, 0.9, 0.7, 0.2, 1) # Last stop, 100% opacity > > ctx.rectangle (0, 0, 1, 1) # Rectangle(x0, y0, x1, y1) > ctx.set_source (pat) > ctx.fill () > > ctx.translate (0.1, 0.1) # Changing the current transformation matrix > > ctx.move_to (0, 0) > ctx.arc (0.2, 0.1, 0.1, -math.pi/2, 0) # Arc(cx, cy, radius, start_angle, stop_angle) > ctx.line_to (0.5, 0.1) # Line to (x,y) > ctx.curve_to (0.5, 0.2, 0.5, 0.4, 0.2, 0.8) # Curve(x1, y1, x2, y2, x3, y3) > ctx.close_path () > > ctx.set_source_rgb (0.3, 0.2, 0.5) # Solid color > ctx.set_line_width (0.02) > ctx.stroke () > > surface.write_to_png ("example.png") # Output to PNG > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Tue Feb 3 17:19:30 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 09:19:30 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 4, 2015 at 4:40 AM, Steven D'Aprano wrote: > given that the glossary need not be 100% complete and definitive, "function > defined inside a class body" is close enough to the truth. * This * We are arguing, not about an element in a formal grammar, but about a glossary entry. If one of my Python students asks me, "What's a method?", I'm not going to go into a technical explanation like this; I want to answer with a single sentence that covers the bit that matters. (Though I'd probably define it from the other perspective - it's "an object attribute that you can call", perhaps - but if the question came from a class definition, "a function defined inside a class" would be fine.) ChrisA From rosuav at gmail.com Tue Feb 3 17:29:37 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 09:29:37 +1100 Subject: Idiomatic backtracking in Python In-Reply-To: <54D13A92.5060701@davea.name> References: <87bnlml44b.fsf@elektro.pacujo.net> <54D13A92.5060701@davea.name> Message-ID: On Wed, Feb 4, 2015 at 8:16 AM, Dave Angel wrote: >> Obviously you have to define the branch somehow, but there are plenty >> of times when you might want to break out of "everything up to here". >> How do you define that? How do you return lots of levels all at once? >> I remember facing this exact problem in trying to solve a particular >> piece-layout puzzle; if I discovered an impossible situation, I could >> actually discard at least two or three levels of recursion all at >> once, because there's no way that the situation could become >> un-impossible within those levels. Can't remember how I ended up >> dealing with that... I think I got naughty and used a global variable. >> > > When I've done things like that, there was no need to do a "return > multiple". You just return, and your caller happens to be a the end of his > loop, so he returns also. > > Classic example of this is the 8 queens puzzle. Each level is going to > examine one row, and once there are no places that aren't yet attacked, it > simply returns. That's true of most problems. I may have to go dig up the code I had, but I believe the basic structure was something along the lines of "place piece in available space, recurse to place piece in each remaining available space". Sometimes, attempting to fill one space would prove that the piece that _created_ that space couldn't possibly go there, so it would do a jump back up a few levels of recursion. Definitely an unusual situation, but not impossible. ChrisA From greg.ewing at canterbury.ac.nz Tue Feb 3 17:43:48 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 11:43:48 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <87egq6adm1.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > Right now Python generates the trampoline from the class prototype every > time you call a method. If the semantics allowed, you could create the > trampoline at instantiation time (for better or worse). That way, the > problem you seem to be referring to wouldn't materialize. Sorry, I misinterpreted what you were suggesting. You seem to be suggesting an optimisation that pre-creates bound methods when the instance is created. Keeping a cache of bound methods would achieve the same thing without changing the semantics. I think CPython might already be doing that, but I'm not sure. -- Greg From davea at davea.name Tue Feb 3 18:02:52 2015 From: davea at davea.name (Dave Angel) Date: Tue, 03 Feb 2015 18:02:52 -0500 Subject: Problem while reading files from hdfs using python In-Reply-To: <19c7923f-2f26-4752-ac2b-1b6dfc093631@googlegroups.com> References: <19c7923f-2f26-4752-ac2b-1b6dfc093631@googlegroups.com> Message-ID: <54D1539C.204@davea.name> On 01/25/2015 03:23 PM, Shalini Ravishankar wrote: > Hello Everyone, > > I am trying to read(open) and write files in hdfs inside a python script. But having error. Please copy/paste the full error traceback. > Can someone tell me what is wrong here. > > Code (full): sample.py > > #!/usr/bin/python > > > from subprocess import Popen, PIPE > > print "Before Loop" > > cat = Popen(["hadoop", "fs", "-cat", "./sample.txt"], > stdout=PIPE) I don't know anything about hadoop, and when you run it separately, you used different parameters. So you can do a lot towards testing it yourself. Start by running hadoop fs -cat ... from shell to see whether it displays anything. You should be able to use exactly the same arguments as you use in the Popen call. Then if that seems to work as you expect, comment out your 'put' code below, and add some prints to the loop. Does that look reasonable? At that point, if both look reasonable, then try the inverse. Write some known data to the 'put' command, and see if it makes it into the appropriate file. Once again, you should be able to also test the program parameters and behavior from the shell, typing manually into stdin. > put = Popen(["hadoop", "fs", "-put", "-", "./modifiedfile.txt"], > stdin=PIPE) > for line in cat.stdout: > line += "Blah" > print line > put.stdin.write(line) > > cat.stdout.close() > cat.wait() > put.stdin.close() > put.wait() > > When I execute : > > hadoop jar /usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.5.1.jar -file ./sample.py -mapper './sample.py' -input sample.txt -output fileRead > > It executes properly I couldn't find the file which supposed to create in hdfs modifiedfile > > And When I execute : > > hadoop fs -getmerge ./fileRead/ file.txt > > Inside the file.txt, I got : > > Before Loop > Before Loop > > Can someone please tell me what I am doing wrong here ?? I dont think it reads from the sample.txt > > I would really appreciate the help. > > > -- > Thanks & Regards, > Shalini Ravishankar. > -- DaveA From marko at pacujo.net Tue Feb 3 18:32:14 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 04 Feb 2015 01:32:14 +0200 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> Message-ID: <878ugea7wh.fsf@elektro.pacujo.net> Gregory Ewing : > You seem to be suggesting an optimisation that pre-creates bound > methods when the instance is created. Keeping a cache of bound methods > would achieve the same thing without changing the semantics. I think > CPython might already be doing that, but I'm not sure. No, I'm saying Python should behave differently. Python: >>> class A: ... def f(self): ... print("f") ... def g(self): ... print("g") ... >>> a = A() >>> a.__class__.f = a.__class__.g >>> a.f() g In my preferred semantics, a.f() would print >>> a.f() f Marko From rosuav at gmail.com Tue Feb 3 18:39:41 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Feb 2015 10:39:41 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <878ugea7wh.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> <878ugea7wh.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 4, 2015 at 10:32 AM, Marko Rauhamaa wrote: > No, I'm saying Python should behave differently. > > Python: > > >>> class A: > ... def f(self): > ... print("f") > ... def g(self): > ... print("g") > ... > >>> a = A() > >>> a.__class__.f = a.__class__.g > >>> a.f() > g > > In my preferred semantics, a.f() would print > > >>> a.f() > f Yeeeouch. So either it has to actually copy everything in on instantiation (stupid cost for the tiny chance that it'll actually ever matter), or else have some kind of versioning that means that it knows that 'a' was created from the pre-changed class. What's the advantage?!? ChrisA From breamoreboy at yahoo.co.uk Tue Feb 3 18:41:39 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 03 Feb 2015 23:41:39 +0000 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <878ugea7wh.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> <878ugea7wh.fsf@elektro.pacujo.net> Message-ID: On 03/02/2015 23:32, Marko Rauhamaa wrote: > Gregory Ewing : > >> You seem to be suggesting an optimisation that pre-creates bound >> methods when the instance is created. Keeping a cache of bound methods >> would achieve the same thing without changing the semantics. I think >> CPython might already be doing that, but I'm not sure. > > No, I'm saying Python should behave differently. > > Python: > > >>> class A: > ... def f(self): > ... print("f") > ... def g(self): > ... print("g") > ... > >>> a = A() > >>> a.__class__.f = a.__class__.g > >>> a.f() > g > > In my preferred semantics, a.f() would print > > >>> a.f() > f IMHO as clear as mud. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ethan at stoneleaf.us Tue Feb 3 18:55:14 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 03 Feb 2015 15:55:14 -0800 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <878ugea7wh.fsf@elektro.pacujo.net> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> <878ugea7wh.fsf@elektro.pacujo.net> Message-ID: <54D15FE2.9060909@stoneleaf.us> On 02/03/2015 03:32 PM, Marko Rauhamaa wrote: > Gregory Ewing : > >> You seem to be suggesting an optimisation that pre-creates bound >> methods when the instance is created. Keeping a cache of bound methods >> would achieve the same thing without changing the semantics. I think >> CPython might already be doing that, but I'm not sure. > > No, I'm saying Python should behave differently. > > Python: > > >>> class A: > ... def f(self): > ... print("f") > ... def g(self): > ... print("g") > ... > >>> a = A() > >>> a.__class__.f = a.__class__.g > >>> a.f() > g > > In my preferred semantics, a.f() would print > > >>> a.f() > f That is, as you acknowledge, not Python. Thankfully, it will also never be Python. However, because Python is so awesome, you can twist your own code to behave that way, to a point: simply have your __init__ ( or __new__) populate the instance dict with all non-dunder methods. Or even better, implement your own proto(mumble) type stack in Python, so there is some warning that your instances don't behave quite like normal Python instances. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From steve+comp.lang.python at pearwood.info Tue Feb 3 20:57:12 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 04 Feb 2015 12:57:12 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> Message-ID: <54d17c7a$0$2890$c3e8da3$76491128@news.astraweb.com> Gregory Ewing wrote: > Marko Rauhamaa wrote: >> For (almost) all practical purposes, that is the Python way as well. If >> object instantiation (conceptually) copied the class's methods into the >> object's dict, you'd get the semantics I'm looking for. > > If things worked the way you want, it would be > impossible to store a function in an instance > attribute and get it out again *without* it > being treated as a method and getting 'self' > added to its arguments. That would be a > considerable nuisance when dealing with > callbacks and the like. Not impossible, just inconvenient. Assuming that the descriptor protocol runs on access to instance attributes as well as class attribute, the solution is to use a custom descriptor to return the bare function. class function(object): def __init__(self, func): self.func = func def __get__(self, obj, cls=None): return self.func instance.callback = function(mycallback) Or you can possibly use staticmethod. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 3 21:13:58 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 04 Feb 2015 13:13:58 +1100 Subject: Ghost vulnerability References: <75fe0f21-3ffb-4649-ad06-0dcbdad631fa@googlegroups.com> <54d0aeb9$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d18066$0$12993$c3e8da3$5496439d@news.astraweb.com> Anssi Saari wrote: > Steven D'Aprano writes: > >> Here's the one-liner: >> >> python -c 'import socket;y="0"*50000000;socket.gethostbyname(y)' >> >> >> I think it is likely that y="0"*50000000 would segfault due to lack of >> memory on many machines. I wouldn't trust this as a test. > > Hmm, how much RAM does that one-liner actually need? My router has 128 > MB total RAM with about 90 MB free. So it can store the string once but > if it's copied with the gethostbyname call then it'll run out... In Python 2, "0" is a byte (plus object header), so about 50MB give or take. In Python 3, "0" is a unicode string, so depending on whether you have a "narrow" or "wide" build, or version 3.3+, that could take 1, 2 or 4 bytes per character. So potentially 50, 100 or 200MB (plus a few extra bytes for the object header). > Anyways, here's an example calling gethostbyname directly in python: > > from ctypes import CDLL > o = CDLL('libc.so.6') > for i in range(0, 2500): > o.gethostbyname('0'*i) > > I don't have a vulnerable system to test on any more though. I stuck a "print i" just before the call to gethostbyname, and it got to i = 1004 and then crashed: *** glibc detected *** python2.7: realloc(): invalid next size: 0x08b9a7c0 *** with a page or three of diagnostics. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 3 21:30:17 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 04 Feb 2015 13:30:17 +1100 Subject: dunder-docs (was Python is DOOMED! Again!) References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <87egq6adm1.fsf@elektro.pacujo.net> Message-ID: <54d18439$0$13011$c3e8da3$5496439d@news.astraweb.com> Gregory Ewing wrote: > Marko Rauhamaa wrote: >> Right now Python generates the trampoline from the class prototype every >> time you call a method. If the semantics allowed, you could create the >> trampoline at instantiation time (for better or worse). That way, the >> problem you seem to be referring to wouldn't materialize. > > Sorry, I misinterpreted what you were suggesting. > > You seem to be suggesting an optimisation that pre-creates > bound methods when the instance is created. Keeping a > cache of bound methods would achieve the same thing > without changing the semantics. I think CPython > might already be doing that, but I'm not sure. It's not. py> class K(object): ... def f(self): pass ... py> k = K() py> k.f is k.f False -- Steve From rantingrickjohnson at gmail.com Tue Feb 3 21:31:59 2015 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 3 Feb 2015 18:31:59 -0800 (PST) Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ca583e$0$13005$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tuesday, February 3, 2015 at 4:05:57 PM UTC-6, Ian wrote: > On Tue, Feb 3, 2015 at 10:40 AM, Steven D'Aprano wrote: > > AT LONG LAST THE LIGHT DAWNS! THE PENNY DROPS! > > Careful there, you're starting to sound like Ranting Rick. ;-) Ha! My meme's are far more catchy and intellectual. But as they say, intimation *IS* the ultimate form of flattery! From w3tmb1 at gmail.com Tue Feb 3 21:52:15 2015 From: w3tmb1 at gmail.com (w3tmb1 at gmail.com) Date: Tue, 3 Feb 2015 18:52:15 -0800 (PST) Subject: re.findall help Message-ID: <11aa8a80-bfdc-4e6f-a922-cb64c4da0466@googlegroups.com> I am trying to extract the following from a data stream using find all what would be the best way to capture the ip address only from the following text " ip=192.168.1.36 port=4992 " I also want to make sure the program can handle the ip that is as high as 255.255.255.255 Thanks for any help you can provide From cs at zip.com.au Tue Feb 3 22:07:20 2015 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 4 Feb 2015 14:07:20 +1100 Subject: re.findall help In-Reply-To: <11aa8a80-bfdc-4e6f-a922-cb64c4da0466@googlegroups.com> References: <11aa8a80-bfdc-4e6f-a922-cb64c4da0466@googlegroups.com> Message-ID: <20150204030720.GA3727@cskk.homeip.net> On 03Feb2015 18:52, w3tmb1 at gmail.com wrote: >I am trying to extract the following from a data stream using find all what >would be the best way to capture the ip address only from the following text " >ip=192.168.1.36 port=4992 " I also want to make sure the program can handle >the ip that is as high as 255.255.255.255 I would not be using re.findall. If you have strings such as the one you describe I would: - call .split() on the string to break it up on whitespace - select the strings starting with "ip=" - split those strings on "=" and grab the stuff after the "=" - if you want to validate the address: split it on '.', check that there are 4 components, call "int()" on each component to check that is an int, and then check the value of the int as being in the range 0..255 inclusive Write some code doing the above, test it, and if stuck, return with: - the code - example input for which it does not work - an explaination of what it does do (including an output transcript showing the badness if possible) - an explaination of how the output is wrong, and what it should look like Cheers, Cameron Simpson From greg.ewing at canterbury.ac.nz Wed Feb 4 01:04:53 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 19:04:53 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d17c7a$0$2890$c3e8da3$76491128@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <87zj8vgtzk.fsf@elektro.pacujo.net> <87vbjjgrpj.fsf@elektro.pacujo.net> <54d17c7a$0$2890$c3e8da3$76491128@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > Gregory Ewing wrote: > >>If things worked the way you want, it would be >>impossible to store a function in an instance >>attribute and get it out again *without* it >>being treated as a method > > Not impossible, just inconvenient... the > solution is to use a custom descriptor But then it's not a plain function any more. Obviously you can wrap your function in something else, but that's the "nuisance" I was talking about. -- Greg From greg.ewing at canterbury.ac.nz Wed Feb 4 01:22:24 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 19:22:24 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d1842c$0$13011$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> <54d1842c$0$13011$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > I'm arguing with those who insist that objects of type MethodType aren't > methods, and objects of type FunctionType aren't functions but methods, > except when they are, based on that simplified, incomplete glossary entry. I'm not arguing that based on the glossary entry. I'm arguing it based on my experience of how the term 'method' is used in the Python community. > I can inject methods into a class or even an instance. If you > think that methods *must* be defined in the class body, you're missing out > on a very powerful technique. I'm willing to accept 'method' as a legitimate term for a function injected into a class body, *if it's used as a method* once it gets there. This is what you don't seem to understand. It's not about the type of the object. You can't write a piece of code to test whether a given object is a method or not, because it's not an intrinsic property of the object. It's a matter of how it's *used*. -- Greg From greg.ewing at canterbury.ac.nz Wed Feb 4 01:22:36 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Feb 2015 19:22:36 +1300 Subject: dunder-docs (was Python is DOOMED! Again!) In-Reply-To: <54d1a6f5$0$11096$c3e8da3@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54ccc2fc$0$13009$c3e8da3$5496439d@news.astraweb.com> <54cd9a7a$0$12984$c3e8da3$5496439d@news.astraweb.com> <54ce526a$0$13012$c3e8da3$5496439d@news.astraweb.com> <54cf242d$0$12991$c3e8da3$5496439d@news.astraweb.com> <54cf6836$0$12996$c3e8da3$5496439d@news.astraweb.com> <54cf849d$0$13005$c3e8da3$5496439d@news.astraweb.com> <54d10807$0$13002$c3e8da3$5496439d@news.astraweb.com> <54d1842c$0$13011$c3e8da3$5496439d@news.astraweb.com> <54d1a6f5$0$11096$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > Er, perhaps "code injection" is not the best name for this, on account of it > also being the name for a security vulnerability anti-pattern: > > I'm talking about a variety of dependency injection where you either add an > entirely new method to an instance, or give the instance it's own custom > method overriding the one declared in the class. I think the term you're after is "monkeypatching". -- Greg From motoom at xs4all.nl Wed Feb 4 02:58:10 2015 From: motoom at xs4all.nl (Michiel Overtoom) Date: Wed, 4 Feb 2015 08:58:10 +0100 Subject: Cairo module In-Reply-To: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> References: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> Message-ID: <2B103F07-590F-4D59-8BD3-C1647C7947C3@xs4all.nl> Hi Poul, I recently used cairo in a python project (https://github.com/luismqueral/jumpcityrecords). To see the cairo drawing directly on the screen I wrote a minimal Gtk application. It's in the 'src' directory and is called 'randomdraw.py'. Maybe it is of some help to you. Greetings, -- "You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes From karenbryain at hotmail.com Wed Feb 4 03:37:46 2015 From: karenbryain at hotmail.com (andrewbryain) Date: Wed, 4 Feb 2015 00:37:46 -0800 (PST) Subject: Best Choose The Best Weight Loss Products Message-ID: <1423039066952-5085200.post@n6.nabble.com> Herbal X Direct Cleanse includes a lot of materials. Its substances include: calcium, chromium, potassium, Garcinia Cambogia, Gymnema sylvestre leaf extracat, Glucomannan, Alpha lipoic acid, willow bark extract, L-Carnitine, Hydroxy Tea, Green Tea leaf extract, caffeine, and Guarana extract. The materials contain: magnesium Stearate, gelatin, silica. Herbal X Direct Cleanse stirring the thyroid gland it can bring about Weight Loss. This way, the metabolic rate within the body will accelerate then burn the fat. For those who wished to try this, it should not be mixed by them with anything since it could cause rashes and diarrhea, and in addition maintain a zero fat diet. There are lots of available in preparing your fat loss instruments to assist you. With some dedication and the proper program losing weight. http://leorskincareblog.com/herbal-x-direct-cleanse/ -- View this message in context: http://python.6.x6.nabble.com/Best-Choose-The-Best-Weight-Loss-Products-tp5085200.html Sent from the Python - python-list mailing list archive at Nabble.com. From mal at python.org Wed Feb 4 04:20:50 2015 From: mal at python.org (M.-A. Lemburg) Date: Wed, 04 Feb 2015 10:20:50 +0100 Subject: PSF Python Brochure "sold out" - help us kick start the second print run ! Message-ID: <54D1E472.3080508@python.org> [Please help spread the word by forwarding to other relevant mailing lists, user groups, etc. world-wide; thanks :-)] ________________________________________________________________________ *** PSF Python Brochure "sold out" *** Please help us kick start the second print run ! http://brochure.getpython.info/ You can also read this posting online at: http://pyfound.blogspot.de/2015/02/psf-python-brochure-sold-out-help-us.html ________________________________________________________________________ First print run distributed to the community We are happy to announce that we have successfully distributed the first 5,000 copies of the PSF Python Brochure to Python conferences and user groups around the world: PSF Python Brochure Vol. 1 http://brochure.getpython.info/brochure-vol1.pdf Even without doing any major announcement of the possibility to order the brochure for conferences and user groups, we easily received enough requests to have the first print run completely distributed in just a few months. Brochures were sent to PyCon US 2014 in Montreal, EuroPython 2014, PyCons and open source conferences in India, UK, South Africa, Australia, New Zealand, France, Belgium and to many professional users around the world. ________________________________________________________________________ Promoting Python to new audiences The feedback we received was positive all around. Conference attendees were really happy to be able to easily show and prove how Python changes the world, to make the point that learning and using Python is a good investment. The brochure helps them in promoting Python in their local and professional communities, especially to the many non-technical people we cannot easily reach with our python.org web site. We would like to thank all our sponsors and contributors for their hard work to achieve this impressing result. Learn more about the project http://brochure.getpython.info/learn-more ________________________________________________________________________ Please help us kick start the second print run ! In order to continue with the distribution of the remaining 5,000 copies of the Vol. 1 edition, we are now looking for two additional half page ad sponsors to help finance the print run and distribution costs. We have reserved the notes area on the inner back cover page of the brochure for these extra ads. If you would like to show your support for Python and its community and reach out to thousands of people interested in Python, now is the time to sign up as ad sponsor! Sign up as ad sponsor http://brochure.getpython.info/ ________________________________________________________________________ Pre-order the PSF Python Brochure We provide two options for pre-ordering the brochure from the second print run which will be available early in April 2015: * free Community Orders for conference and user groups * paid Company Orders for companies and organizations The costs for the community orders are sponsored through sponsor ads, the PSF and the company orders. Place your pre-order http://brochure.getpython.info/ ________________________________________________________________________ More information More information on the brochure, the idea and people behind it, media data and ordering links are available on our project page: PSF Python Brochure Project Website http://brochure.getpython.info/ Thank you for your help, -- Marc-Andre Lemburg Director Python Software Foundation http://www.python.org/psf/ From jesseporeddu at aim.com Wed Feb 4 06:58:18 2015 From: jesseporeddu at aim.com (VOGLIAMO RENZUSCONI IN GALERA!) Date: Wed, 4 Feb 2015 03:58:18 -0800 (PST) Subject: -- redacted -- Message-ID: <81aa177b-74fe-4ddd-a417-1106342f52e9@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:21:44 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:21:44 -0800 (PST) Subject: -- redacted -- In-Reply-To: <81aa177b-74fe-4ddd-a417-1106342f52e9@googlegroups.com> References: <81aa177b-74fe-4ddd-a417-1106342f52e9@googlegroups.com> Message-ID: <87aeba5b-c1b0-4233-a926-4c593ad7b12d@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:22:06 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:22:06 -0800 (PST) Subject: -- redacted -- In-Reply-To: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: <19b47dd4-bf85-414f-9438-019bb2bbd0a3@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:22:23 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:22:23 -0800 (PST) Subject: -- redacted -- In-Reply-To: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> References: <9ea713a4-3e21-4522-b751-5d896c5ea31e@googlegroups.com> Message-ID: <1509060e-fc21-4f89-a7f3-c2c1ddff94f1@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:22:37 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:22:37 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <945df074-47ee-404b-bffb-93640e66df6e@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:22:51 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:22:51 -0800 (PST) Subject: -- redacted -- In-Reply-To: <04938df1-8d59-4262-acff-648c1475d0f5@googlegroups.com> References: <04938df1-8d59-4262-acff-648c1475d0f5@googlegroups.com> Message-ID: <9fafe23f-46e0-4a94-82e5-2e4493a5da67@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Wed Feb 4 07:24:58 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Wed, 4 Feb 2015 04:24:58 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <1a005766-2f1a-49a4-8996-7fed4dbe5528@googlegroups.com> -- redacted -- From ndbecker2 at gmail.com Wed Feb 4 08:23:40 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Wed, 04 Feb 2015 08:23:40 -0500 Subject: basic generator question Message-ID: I have an object that expects to call a callable to get a value: class obj: def __init__ (self, gen): self.gen = gen def __call__ (self): return self.gen() Now I want gen to be a callable that repeats N times. I'm thinking, this sounds perfect for yield class rpt: def __init__ (self, value, rpt): self.value = value; self.rpt = rpt def __call__ (self): for i in range (self.rpt): yield self.value so I would do: my_rpt_obj = obj (rpt ('hello', 5)) to repeat 'hello' 5 times (for example). But this doesn't work. when obj calls self.gen(), that returns a generator, not the next value. How can I make this work? I can't change the interface of the existing class obj, which expects a callable to get the next value. -- -- Those who don't understand recursion are doomed to repeat it From jugurtha.hadjar at gmail.com Wed Feb 4 08:35:02 2015 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Wed, 04 Feb 2015 14:35:02 +0100 Subject: re.findall help In-Reply-To: <11aa8a80-bfdc-4e6f-a922-cb64c4da0466@googlegroups.com> References: <11aa8a80-bfdc-4e6f-a922-cb64c4da0466@googlegroups.com> Message-ID: <54D22006.5080702@gmail.com> On 02/04/2015 03:52 AM, w3tmb1 at gmail.com wrote: > I am trying to extract the following from a data stream using find > all what would be the best way to capture the ip address only from > the following text " ip=192.168.1.36 port=4992 " I also want to make > sure the program can handle the ip that is as high as > 255.255.255.255 > > Thanks for any help you can provide > Hello, It depends on whether you trust the data (if it's yours and you *know* they're IP addresses) and just want to automate, or not.. >>> pattern = re.compile(r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}') Tested on: >>> test = "ip=192.168.1.36 port=4992 ip=255.255.255.255 port=80" Gives: ['192.168.1.36', '255.255.255.255'] Add "ip=" in order to avoid confusion (data may have by chance something that looks like an IP address, but the "ip=" bit acts as a discriminant. >>> pattern = re.compile(r'ip=\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}') Testing for the string test: test = "ip=192.168.1.36 port=4992fdjsqklmfqsjdkip=192.168.541.36 port=222 2.2.2.2random" Gives: ['ip=192.168.1.36', 'ip=192.168.541.36'] It ignores the 2.2.2.2 because it doesn't have "ip=" in front of it, and was just lucky to have an IP address structure. You can then split "ip=" out of each item to have the IP address. -- ~Jugurtha Hadjar, From nomail at invalid.com Wed Feb 4 09:08:42 2015 From: nomail at invalid.com (ast) Date: Wed, 4 Feb 2015 15:08:42 +0100 Subject: meaning of: line, = Message-ID: <54d227ef$0$3292$426a74cc@news.free.fr> hello I dont understand why there is a comma just after line in the following command: line, = plt.plot(x, np.sin(x), '--', linewidth=2) I never saw that before Found here: http://matplotlib.org/examples/lines_bars_and_markers/line_demo_dash_control.html thanks From kirotawa at gmail.com Wed Feb 4 09:22:30 2015 From: kirotawa at gmail.com (leo kirotawa) Date: Wed, 4 Feb 2015 12:22:30 -0200 Subject: meaning of: line, = In-Reply-To: <54d227ef$0$3292$426a74cc@news.free.fr> References: <54d227ef$0$3292$426a74cc@news.free.fr> Message-ID: You'll find some explanation here: http://stackoverflow.com/questions/1708292/meaning-of-using-commas-and-underscores-with-python-assignment-operator On Wed, Feb 4, 2015 at 12:08 PM, ast wrote: > hello > > I dont understand why there is a comma just after line in the following > command: > > line, = plt.plot(x, np.sin(x), '--', linewidth=2) > > > I never saw that before > > Found here: > http://matplotlib.org/examples/lines_bars_and_markers/line_demo_dash_control.html > > thanks > -- > https://mail.python.org/mailman/listinfo/python-list -- ---------------------------------------------- Le?nidas S. Barbosa (Kirotawa) blog: corecode.wordpress.com From rosuav at gmail.com Wed Feb 4 09:24:52 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Feb 2015 01:24:52 +1100 Subject: meaning of: line, = In-Reply-To: <54d227ef$0$3292$426a74cc@news.free.fr> References: <54d227ef$0$3292$426a74cc@news.free.fr> Message-ID: On Thu, Feb 5, 2015 at 1:08 AM, ast wrote: > I dont understand why there is a comma just after line in the following > command: > > line, = plt.plot(x, np.sin(x), '--', linewidth=2) > > > I never saw that before > > Found here: > http://matplotlib.org/examples/lines_bars_and_markers/line_demo_dash_control.html > That's a slightly unusual form of unpacking. Compare: def get_values(): return 5, 7, 2 x, y, z = get_values() This is like "x = 5; y = 7; z = 2", because it unpacks the function's return value into those three targets. What you have is exactly the same, except that it has only one target. So it's expecting plt.plot() to return an iterable with exactly one thing in it, and it'll unpack it and put that thing into line: def get_packaged_value(): return [42] x, = get_packaged_value() This is equivalent to "x = 42". I don't know matplotlib, so I don't know what it's returning or why, but as long as it's iterable and yields exactly one thing, this will work. ChrisA From rosuav at gmail.com Wed Feb 4 09:32:28 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Feb 2015 01:32:28 +1100 Subject: basic generator question In-Reply-To: References: Message-ID: On Thu, Feb 5, 2015 at 12:23 AM, Neal Becker wrote: > Now I want gen to be a callable that repeats N times. I'm thinking, this > sounds perfect for yield > > class rpt: > def __init__ (self, value, rpt): > self.value = value; self.rpt = rpt > def __call__ (self): > for i in range (self.rpt): > yield self.value > > so I would do: > > my_rpt_obj = obj (rpt ('hello', 5)) > > to repeat 'hello' 5 times (for example). > > But this doesn't work. when obj calls self.gen(), that returns a generator, not > the next value. > > How can I make this work? I can't change the interface of the existing class > obj, which expects a callable to get the next value. So, if I understand you correctly, you want your rpt object to return 'hello' five times to five consecutive calls? >>> a = rpt() >>> a() 'hello' >>> a() 'hello' >>> a() 'hello' >>> a() 'hello' >>> a() 'hello' You could do that with a generator by repeatedly calling next() on it, or you could just keep track of the number of times you were called: class rpt: def __init__ (self, value, rpt): self.value = value; self.rpt = rpt def __call__ (self): if self.rpt: self.rpt -= 1 return self.value # ... otherwise? Up to you to figure out what to do when self.rpt hits zero. ChrisA From fomcl at yahoo.com Wed Feb 4 09:38:31 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 4 Feb 2015 14:38:31 +0000 (UTC) Subject: meaning of: line, = In-Reply-To: References: Message-ID: <657497900.499606.1423060711538.JavaMail.yahoo@mail.yahoo.com> ----- Original Message ----- > From: Chris Angelico > To: > Cc: "python-list at python.org" > Sent: Wednesday, February 4, 2015 3:24 PM > Subject: Re: meaning of: line, = > > On Thu, Feb 5, 2015 at 1:08 AM, ast wrote: >> I dont understand why there is a comma just after line in the following >> command: >> >> line, = plt.plot(x, np.sin(x), '--', linewidth=2) >> >> >> I never saw that before >> >> Found here: >> > http://matplotlib.org/examples/lines_bars_and_markers/line_demo_dash_control.html >> > > That's a slightly unusual form of unpacking. Compare: > > def get_values(): > return 5, 7, 2 > > x, y, z = get_values() > > This is like "x = 5; y = 7; z = 2", because it unpacks the > function's > return value into those three targets. > > What you have is exactly the same, except that it has only one target. > So it's expecting plt.plot() to return an iterable with exactly one > thing in it, and it'll unpack it and put that thing into line: > > def get_packaged_value(): > return [42] > > x, = get_packaged_value() > > This is equivalent to "x = 42". I don't know matplotlib, so I > don't > know what it's returning or why, but as long as it's iterable and > yields exactly one thing, this will work. I have also never seen this before, but perhaps this: >>> f = lambda: [42] >>> result, = f() >>> result 42 ... is slightly cleaner than this: >>> result = f()[0] >>> result 42 From rosuav at gmail.com Wed Feb 4 10:04:26 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Feb 2015 02:04:26 +1100 Subject: meaning of: line, = In-Reply-To: <657497900.499606.1423060711538.JavaMail.yahoo@mail.yahoo.com> References: <657497900.499606.1423060711538.JavaMail.yahoo@mail.yahoo.com> Message-ID: On Thu, Feb 5, 2015 at 1:38 AM, Albert-Jan Roskam wrote: > I have also never seen this before, but perhaps this: > >>>> f = lambda: [42] >>>> result, = f() >>>> result > 42 > > ... is slightly cleaner than this: >>>> result = f()[0] >>>> result > 42 They're not technically identical. If the thing returned is subscriptable (as with your list example), then I would definitely subscript it rather than unpacking; but if it's something iterable but not subscriptable, the unpack will still work. >>> def f(): yield 42 ... >>> result, = f() >>> result 42 >>> result = f()[0] Traceback (most recent call last): File "", line 1, in TypeError: 'generator' object is not subscriptable ChrisA From rustompmody at gmail.com Wed Feb 4 10:09:54 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 4 Feb 2015 07:09:54 -0800 (PST) Subject: meaning of: line, = In-Reply-To: References: Message-ID: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> On Wednesday, February 4, 2015 at 8:14:29 PM UTC+5:30, Albert-Jan Roskam wrote: > ----- Original Message ----- > > > From: Chris Angelico > > Sent: Wednesday, February 4, 2015 3:24 PM > > Subject: Re: meaning of: line, = > > > > On Thu, Feb 5, 2015 at 1:08 AM, ast wrote: > >> I dont understand why there is a comma just after line in the following > >> command: > >> > >> line, = plt.plot(x, np.sin(x), '--', linewidth=2) > >> > >> > >> I never saw that before > >> > >> Found here: > >> > > http://matplotlib.org/examples/lines_bars_and_markers/line_demo_dash_control.html > >> > > > > That's a slightly unusual form of unpacking. Compare: > > > > def get_values(): > > return 5, 7, 2 > > > > x, y, z = get_values() > > > > This is like "x = 5; y = 7; z = 2", because it unpacks the > > function's > > return value into those three targets. > > > > What you have is exactly the same, except that it has only one target. > > So it's expecting plt.plot() to return an iterable with exactly one > > thing in it, and it'll unpack it and put that thing into line: > > > > def get_packaged_value(): > > return [42] > > > > x, = get_packaged_value() > > > > This is equivalent to "x = 42". I don't know matplotlib, so I > > don't > > know what it's returning or why, but as long as it's iterable and > > yields exactly one thing, this will work. > > > > I have also never seen this before, but perhaps this: > > >>> f = lambda: [42] > >>> result, = f() > >>> result > 42 > > ... is slightly cleaner than this: > >>> result = f()[0] > >>> result > 42 Well its cryptic and confusing (to me at least) And is helped by adding 2 characters: (result,) = f() instead of result, = f() From joel.goldstick at gmail.com Wed Feb 4 10:19:34 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 4 Feb 2015 10:19:34 -0500 Subject: basic generator question In-Reply-To: References: Message-ID: On Wed, Feb 4, 2015 at 9:32 AM, Chris Angelico wrote: > On Thu, Feb 5, 2015 at 12:23 AM, Neal Becker wrote: > > Now I want gen to be a callable that repeats N times. I'm thinking, this > > sounds perfect for yield > > > > class rpt: > > def __init__ (self, value, rpt): > > self.value = value; self.rpt = rpt > > def __call__ (self): > > for i in range (self.rpt): > > yield self.value > > > > so I would do: > > > > my_rpt_obj = obj (rpt ('hello', 5)) > > > > to repeat 'hello' 5 times (for example). > > > > But this doesn't work. when obj calls self.gen(), that returns a > generator, not > > the next value. > > > > How can I make this work? I can't change the interface of the existing > class > > obj, which expects a callable to get the next value. > Can you actually show your code and the traceback? > > So, if I understand you correctly, you want your rpt object to return > 'hello' five times to five consecutive calls? > > >>> a = rpt() > >>> a() > 'hello' > >>> a() > 'hello' > >>> a() > 'hello' > >>> a() > 'hello' > >>> a() > 'hello' > > You could do that with a generator by repeatedly calling next() on it, > or you could just keep track of the number of times you were called: > > class rpt: > def __init__ (self, value, rpt): > self.value = value; self.rpt = rpt > def __call__ (self): > if self.rpt: > self.rpt -= 1 > return self.value > # ... otherwise? > > Up to you to figure out what to do when self.rpt hits zero. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.goldstick at gmail.com Wed Feb 4 10:25:41 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 4 Feb 2015 10:25:41 -0500 Subject: basic generator question In-Reply-To: References: Message-ID: On Wed, Feb 4, 2015 at 10:19 AM, Joel Goldstick wrote: > > > On Wed, Feb 4, 2015 at 9:32 AM, Chris Angelico wrote: > >> On Thu, Feb 5, 2015 at 12:23 AM, Neal Becker wrote: >> > Now I want gen to be a callable that repeats N times. I'm thinking, >> this >> > sounds perfect for yield >> > >> > class rpt: >> > def __init__ (self, value, rpt): >> > self.value = value; self.rpt = rpt >> > def __call__ (self): >> > for i in range (self.rpt): >> > yield self.value >> > >> > so I would do: >> > >> > my_rpt_obj = obj (rpt ('hello', 5)) >> > >> > to repeat 'hello' 5 times (for example). >> > >> > But this doesn't work. when obj calls self.gen(), that returns a >> generator, not >> > the next value. >> > >> > How can I make this work? I can't change the interface of the existing >> class >> > obj, which expects a callable to get the next value. >> > > Can you actually show your code and the traceback? > When you do rpt('hello',5)) you create an instance of rpt but you don't actually invoke it. maybe try: my_rpt_obj = obj(rpt('hello',5))() If I'm thinking straight, its the final () that actually cause your __call__ to happen. > >> So, if I understand you correctly, you want your rpt object to return >> 'hello' five times to five consecutive calls? >> >> >>> a = rpt() >> >>> a() >> 'hello' >> >>> a() >> 'hello' >> >>> a() >> 'hello' >> >>> a() >> 'hello' >> >>> a() >> 'hello' >> >> You could do that with a generator by repeatedly calling next() on it, >> or you could just keep track of the number of times you were called: >> >> class rpt: >> def __init__ (self, value, rpt): >> self.value = value; self.rpt = rpt >> def __call__ (self): >> if self.rpt: >> self.rpt -= 1 >> return self.value >> # ... otherwise? >> >> Up to you to figure out what to do when self.rpt hits zero. >> >> ChrisA >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > > > -- > Joel Goldstick > http://joelgoldstick.com > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Feb 4 11:02:24 2015 From: __peter__ at web.de (Peter Otten) Date: Wed, 04 Feb 2015 17:02:24 +0100 Subject: basic generator question References: Message-ID: Neal Becker wrote: > I have an object that expects to call a callable to get a value: > > class obj: > def __init__ (self, gen): > self.gen = gen > def __call__ (self): > return self.gen() As written that looks a bit like if boolean_expression == True: ... as you could replace inst = obj(callable) with inst = callable but that may be an artifact of the example. > Now I want gen to be a callable that repeats N times. I'm thinking, this > sounds perfect for yield > > class rpt: > def __init__ (self, value, rpt): > self.value = value; self.rpt = rpt > def __call__ (self): > for i in range (self.rpt): > yield self.value > > so I would do: > > my_rpt_obj = obj (rpt ('hello', 5)) > > to repeat 'hello' 5 times (for example). What do you expect to happen when my_rpt_obj is called the sixth time? > But this doesn't work. when obj calls self.gen(), that returns a > generator, not the next value. > > How can I make this work? I can't change the interface of the existing > class obj, which expects a callable to get the next value. >>> class Obj: ... def __init__(self, gen): ... self.gen = iter(gen) ... def __call__(self): ... return next(self.gen) ... >>> class Repeat: ... def __init__(self, value, times): ... self.value = value ... self.times = times ... def __iter__(self): ... for i in range(self.times): ... yield self.value ... >>> r = Obj(Repeat("hello", 3)) >>> r() 'hello' >>> r() 'hello' >>> r() 'hello' >>> r() Traceback (most recent call last): File "", line 1, in File "", line 5, in __call__ StopIteration Instead of the Repeat class you may use a generator: >>> def repeat(value, times): ... for i in range(times): ... yield value ... >>> r = Obj(repeat("hello", 2)) >>> r() 'hello' >>> r() 'hello' >>> r() Traceback (most recent call last): File "", line 1, in File "", line 5, in __call__ StopIteration This is for demonstration purposes as there is already itertools.repeat(): >>> import itertools >>> r = Obj(itertools.repeat("world", 2)) >>> r() 'world' >>> r() 'world' >>> r() Traceback (most recent call last): File "", line 1, in File "", line 5, in __call__ StopIteration Depending on your actual need you may also omit the Obj() class: >>> import functools >>> r = functools.partial(next, itertools.repeat("goodbye", 2)) >>> r() 'goodbye' >>> r() 'goodbye' >>> r() Traceback (most recent call last): File "", line 1, in StopIteration Somewhat less formal you can bind the iterator method directly: >>> r = itertools.repeat("GOODBYE", 2).__next__ # next in Python 2 >>> r() 'GOODBYE' >>> r() 'GOODBYE' >>> r() Traceback (most recent call last): File "", line 1, in StopIteration From steve+comp.lang.python at pearwood.info Wed Feb 4 11:09:37 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 05 Feb 2015 03:09:37 +1100 Subject: basic generator question References: Message-ID: <54d24442$0$13001$c3e8da3$5496439d@news.astraweb.com> Neal Becker wrote: > I have an object that expects to call a callable to get a value: > > class obj: > def __init__ (self, gen): > self.gen = gen > def __call__ (self): > return self.gen() As written, there is no need for this "obj" class, it just adds a pointless layer of indirection. Perhaps it was written by a Java programmer? http://steve-yegge.blogspot.com.au/2006/03/execution-in-kingdom-of-nouns.html Instead of: x = obj(something_callable) # later result = x() just do: x = something_callable result = x() or even result = something_callable() > Now I want gen to be a callable that repeats N times. What happens on the sixth time? The world ends? Most callables can be called indefinitely. I'm going to assume an exception. Later I'll use a RuntimeException, but you can do anything you like. > I'm thinking, this sounds perfect for yield > > class rpt: > def __init__ (self, value, rpt): > self.value = value; self.rpt = rpt > def __call__ (self): > for i in range (self.rpt): > yield self.value > > so I would do: > > my_rpt_obj = obj (rpt ('hello', 5)) > > to repeat 'hello' 5 times (for example). > > But this doesn't work. when obj calls self.gen(), that returns a > generator, not the next value. > > How can I make this work? I can't change the interface of the existing > class obj, which expects a callable to get the next value. The built-in function `next` almost does what we want, except the exception raised is inappropriate. Unless you are very careful, you don't want to allow StopIteration to propagate, lest it be caught in an unexpected place and cause surprises. (Not the pleasant kind.) py> from functools import partial py> f = partial(next, rpt("hello", 2)()) # note the extra parens py> f() 'hello' py> f() 'hello' py> f() Traceback (most recent call last): File "", line 1, in StopIteration But I wouldn't use rpt as given. I'd say: def repeat(obj, count): for i in range(count): yield obj raise RuntimeError # or whatever mycallable = partial(next, repeat("hello", 5)) # no extra parens And now pass mycallable to your other class. P.S. your naming conventions give me a blood clot. Class "rpt" taking an argument "rpt" -- I lost six months of life just from reading that. -- Steven From travisgriggs at gmail.com Wed Feb 4 11:50:40 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Wed, 4 Feb 2015 08:50:40 -0800 Subject: pymongo and attribute dictionaries Message-ID: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> I really like pymongo. And I really like Python. But one thing my fingers really get tired of typing is someDoc[?_?id?] This just does not roll of the fingers well. Too many ?reach for modifier keys? in a row. I would rather use someDoc._id Googling shows that I?m not the first to want to do this in the general sense (e.g. http://stackoverflow.com/questions/4984647/accessing-dict-keys-like-an-attribute-in-python). Arguments aside of whether this should or shouldn?t be done, I want to know how I might solve this with Python. Consider it an academic pursuit. The problem I have is not how to do the AttributeDictionary subclass, there are plenty of those examples. The problem is that the pymongo APIs already return dictionaries. In a language (Smalltalk, Objective-C, Ruby) that supports class extensions, that would be my first tool of choice to solve this problem. I?d just extend Dictionary to behave the way I want and be done with it. I can?t do that in Python though. I guess I could make my own module that subclasses the relevant pymongo classes, and do super() calling implementations of all of the relevant methods, coercing the return type. That is a maintenance headache though. What are my options, if any? From ethan at stoneleaf.us Wed Feb 4 12:09:59 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 04 Feb 2015 09:09:59 -0800 Subject: meaning of: line, = In-Reply-To: References: <657497900.499606.1423060711538.JavaMail.yahoo@mail.yahoo.com> Message-ID: <54D25267.8050104@stoneleaf.us> On 02/04/2015 07:04 AM, Chris Angelico wrote: > On Thu, Feb 5, 2015 at 1:38 AM, Albert-Jan Roskam wrote: >> I have also never seen this before, but perhaps this: >> >>>>> f = lambda: [42] >>>>> result, = f() >>>>> result >> 42 >> >> ... is slightly cleaner than this: >>>>> result = f()[0] >>>>> result >> 42 > > They're not technically identical. If the thing returned is > subscriptable (as with your list example), then I would definitely > subscript it rather than unpacking; By unpacking you are also double checking that the returned iterable contains exactly one item; more or less will cause an exception -- you only get half that check if you index. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ian.g.kelly at gmail.com Wed Feb 4 12:15:35 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Feb 2015 10:15:35 -0700 Subject: basic generator question In-Reply-To: References: Message-ID: On Wed, Feb 4, 2015 at 6:23 AM, Neal Becker wrote: > class rpt: > def __init__ (self, value, rpt): > self.value = value; self.rpt = rpt > def __call__ (self): > for i in range (self.rpt): > yield self.value Note that this class is just reimplementing itertools.repeat. >>> list(itertools.repeat('hello', 5)) ['hello', 'hello', 'hello', 'hello', 'hello'] From ian.g.kelly at gmail.com Wed Feb 4 12:22:02 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Feb 2015 10:22:02 -0700 Subject: pymongo and attribute dictionaries In-Reply-To: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> Message-ID: On Wed, Feb 4, 2015 at 9:50 AM, Travis Griggs wrote: > I really like pymongo. And I really like Python. But one thing my fingers really get tired of typing is > > someDoc[?_?id?] > > This just does not roll of the fingers well. Too many ?reach for modifier keys? in a row. I would rather use > > someDoc._id > > Googling shows that I?m not the first to want to do this in the general sense (e.g. http://stackoverflow.com/questions/4984647/accessing-dict-keys-like-an-attribute-in-python). > > Arguments aside of whether this should or shouldn?t be done, I want to know how I might solve this with Python. Consider it an academic pursuit. > > The problem I have is not how to do the AttributeDictionary subclass, there are plenty of those examples. The problem is that the pymongo APIs already return dictionaries. In a language (Smalltalk, Objective-C, Ruby) that supports class extensions, that would be my first tool of choice to solve this problem. I?d just extend Dictionary to behave the way I want and be done with it. I can?t do that in Python though. I guess I could make my own module that subclasses the relevant pymongo classes, and do super() calling implementations of all of the relevant methods, coercing the return type. That is a maintenance headache though. > > What are my options, if any? You could construct the AttributeDictionary by copying the dict returned from pymongo. The question then is whether the copying would be too expensive or not. Alternately, you could just wrap the dictionaries returned by pymongo in an object. Something like this should be all you need: class AttrDictWrapper(object): def __init__(self, the_dict): self.__dict__ = the_dict >>> d = AttrDictWrapper({'spam': 42, 'ham': False}) >>> d.spam 42 >>> d.ham False From __peter__ at web.de Wed Feb 4 12:36:48 2015 From: __peter__ at web.de (Peter Otten) Date: Wed, 04 Feb 2015 18:36:48 +0100 Subject: meaning of: line, = References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> Message-ID: Rustom Mody wrote: > Well its cryptic and confusing (to me at least) > And is helped by adding 2 characters: > > (result,) = f() > > instead of > > result, = f() Another alternative is to put a list literal on the lefthand side: >>> def f(): yield 42 ... >>> [result] = f() >>> result 42 (If you're worried: neither the list nor the tuple will be created; the bytecode is identical in both cases) From travisgriggs at gmail.com Wed Feb 4 15:16:24 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Wed, 4 Feb 2015 12:16:24 -0800 Subject: pymongo and attribute dictionaries In-Reply-To: References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> Message-ID: <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> > On Feb 4, 2015, at 9:22 AM, Ian Kelly wrote: > > On Wed, Feb 4, 2015 at 9:50 AM, Travis Griggs wrote: >> I really like pymongo. And I really like Python. But one thing my fingers really get tired of typing is >> >> someDoc[?_?id?] >> >> This just does not roll of the fingers well. Too many ?reach for modifier keys? in a row. I would rather use >> >> someDoc._id >> >> Googling shows that I?m not the first to want to do this in the general sense (e.g. http://stackoverflow.com/questions/4984647/accessing-dict-keys-like-an-attribute-in-python). >> >> Arguments aside of whether this should or shouldn?t be done, I want to know how I might solve this with Python. Consider it an academic pursuit. >> >> The problem I have is not how to do the AttributeDictionary subclass, there are plenty of those examples. The problem is that the pymongo APIs already return dictionaries. In a language (Smalltalk, Objective-C, Ruby) that supports class extensions, that would be my first tool of choice to solve this problem. I?d just extend Dictionary to behave the way I want and be done with it. I can?t do that in Python though. I guess I could make my own module that subclasses the relevant pymongo classes, and do super() calling implementations of all of the relevant methods, coercing the return type. That is a maintenance headache though. >> >> What are my options, if any? > > You could construct the AttributeDictionary by copying the dict > returned from pymongo. The question then is whether the copying would > be too expensive or not. > > Alternately, you could just wrap the dictionaries returned by pymongo > in an object. Something like this should be all you need: > > class AttrDictWrapper(object): > def __init__(self, the_dict): > self.__dict__ = the_dict > >>>> d = AttrDictWrapper({'spam': 42, 'ham': False}) >>>> d.spam > 42 >>>> d.ham > False > Yes, that is clever. So if you wanted to minimize the amount of typing you had to do at all of your pymongo API call sites, what strategy would you use to keep that relatively terse? Is the following the right approach to take? class Doc(object): def __init__(self, target): self.__dict__ = target and then something like for doc in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): pprint(doc) changes to for doc in ((Doc(d) for d in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): pprint(doc) Are there other approaches? Feel free to impress me with evil abuses in the interest of academic enrichment... From ckaynor at zindagigames.com Wed Feb 4 15:45:51 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Wed, 4 Feb 2015 12:45:51 -0800 Subject: pymongo and attribute dictionaries In-Reply-To: <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> Message-ID: On Wed, Feb 4, 2015 at 12:16 PM, Travis Griggs wrote: > >> On Feb 4, 2015, at 9:22 AM, Ian Kelly wrote: >> >> On Wed, Feb 4, 2015 at 9:50 AM, Travis Griggs wrote: >>> I really like pymongo. And I really like Python. But one thing my fingers really get tired of typing is >>> >>> someDoc[?_?id?] >>> >>> This just does not roll of the fingers well. Too many ?reach for modifier keys? in a row. I would rather use >>> >>> someDoc._id >>> >>> Googling shows that I?m not the first to want to do this in the general sense (e.g. http://stackoverflow.com/questions/4984647/accessing-dict-keys-like-an-attribute-in-python). >>> >>> Arguments aside of whether this should or shouldn?t be done, I want to know how I might solve this with Python. Consider it an academic pursuit. >>> >>> The problem I have is not how to do the AttributeDictionary subclass, there are plenty of those examples. The problem is that the pymongo APIs already return dictionaries. In a language (Smalltalk, Objective-C, Ruby) that supports class extensions, that would be my first tool of choice to solve this problem. I?d just extend Dictionary to behave the way I want and be done with it. I can?t do that in Python though. I guess I could make my own module that subclasses the relevant pymongo classes, and do super() calling implementations of all of the relevant methods, coercing the return type. That is a maintenance headache though. >>> >>> What are my options, if any? >> >> You could construct the AttributeDictionary by copying the dict >> returned from pymongo. The question then is whether the copying would >> be too expensive or not. >> >> Alternately, you could just wrap the dictionaries returned by pymongo >> in an object. Something like this should be all you need: >> >> class AttrDictWrapper(object): >> def __init__(self, the_dict): >> self.__dict__ = the_dict >> >>>>> d = AttrDictWrapper({'spam': 42, 'ham': False}) >>>>> d.spam >> 42 >>>>> d.ham >> False >> > > Yes, that is clever. So if you wanted to minimize the amount of typing you had to do at all of your pymongo API call sites, what strategy would you use to keep that relatively terse? > > Is the following the right approach to take? > > class Doc(object): > def __init__(self, target): > self.__dict__ = target > > and then something like > > for doc in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): > pprint(doc) > > changes to > > for doc in ((Doc(d) for d in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): > pprint(doc) > > Are there other approaches? Feel free to impress me with evil abuses in the interest of academic enrichment... One possible approach, that may or may not work based on the use case, would be to monkey patch the module. The main drawback is that if it is used in places that do not expect the wrapper, it could well break. Untested code: import types def monkey(module): for attr in dir(module): if attr.startswith('_'): # Ignore private attributes. continue object = getattr(module, attr) if not isinstance(object, types.Function): # Ignore non-functions. continue setattr(module, attr, lambda *args, **kwargs: AttrDictWrapper(object(*args, **kwargs))) Another option would be to create a wrapper module. This option has the side-effect that it would be difficult to recurse into classes, as you would have to copy the class. This would be very similar (again, untested): import types # Note that this may get overridden in the function, which could cause problems. def makeWrapper(module): for attr in dir(module): if attr.startswith('_'): # Ignore private attributes. continue object = getattr(module, attr) if not isinstance(object, types.Function): # Ignore non-functions. object = lambda *args, **kwargs: AttrDictWrapper(object(*args, **kwargs)) globals()[attr] = object Both of these are written to (attempt) to replace all instances. Obviously, this could be reduced to a white-list or additional checks could be made to restrict the wrapped sets. I would be VERY weary of actually using the first due to the noted drawback. The second is safer, but you would want to be careful to test it for unexpected side-effects. From greg.ewing at canterbury.ac.nz Wed Feb 4 15:52:31 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 05 Feb 2015 09:52:31 +1300 Subject: pymongo and attribute dictionaries In-Reply-To: References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> Message-ID: Travis Griggs wrote: > for doc in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): pprint(doc) > > changes to > > for doc in ((Doc(d) for d in client.db.radios.find({?_id': {?$regex?: > ?^[ABC]'}})): pprint(doc) > > Are there other approaches? Feel free to impress me with evil abuses in the > interest of academic enrichment... You could encapsulate some of that in a helper function such as def docs(source): for d in source: yield Doc(d) then your example becomes for doc in docs(client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): pprint(doc) -- Greg From ian.g.kelly at gmail.com Wed Feb 4 15:53:28 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Feb 2015 13:53:28 -0700 Subject: pymongo and attribute dictionaries In-Reply-To: <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> Message-ID: On Wed, Feb 4, 2015 at 1:16 PM, Travis Griggs wrote: > Yes, that is clever. So if you wanted to minimize the amount of typing you had to do at all of your pymongo API call sites, what strategy would you use to keep that relatively terse? > > Is the following the right approach to take? > > class Doc(object): > def __init__(self, target): > self.__dict__ = target > > and then something like > > for doc in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): > pprint(doc) > > changes to > > for doc in ((Doc(d) for d in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): > pprint(doc) > > Are there other approaches? Feel free to impress me with evil abuses in the interest of academic enrichment... I'd prefer map (or itertools.imap in Python 2) over the inline generator in this case: for doc in map(Doc, client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): pprint(doc) Or if you like, a utility function wrapping the same. def docs(dicts): return map(Doc, dicts) From ian.g.kelly at gmail.com Wed Feb 4 15:54:22 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Feb 2015 13:54:22 -0700 Subject: pymongo and attribute dictionaries In-Reply-To: <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> Message-ID: On Wed, Feb 4, 2015 at 1:16 PM, Travis Griggs wrote: > Yes, that is clever. So if you wanted to minimize the amount of typing you had to do at all of your pymongo API call sites, what strategy would you use to keep that relatively terse? > > Is the following the right approach to take? > > class Doc(object): > def __init__(self, target): > self.__dict__ = target > > and then something like > > for doc in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): > pprint(doc) > > changes to > > for doc in ((Doc(d) for d in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): > pprint(doc) > > Are there other approaches? Feel free to impress me with evil abuses in the interest of academic enrichment... I'd prefer map (or itertools.imap in Python 2) over the inline generator in this case: for doc in map(Doc, client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}})): pprint(doc) Or if you like, a utility function wrapping the same. def docs(dicts): return map(Doc, dicts) From rosuav at gmail.com Wed Feb 4 16:18:39 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Feb 2015 08:18:39 +1100 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> Message-ID: On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten <__peter__ at web.de> wrote: > Another alternative is to put a list literal on the lefthand side: > >>>> def f(): yield 42 > > ... >>>> [result] = f() >>>> result > 42 Huh, was not aware of that alternate syntax. > (If you're worried: neither the list nor the tuple will be created; the > bytecode is identical in both cases) It can't possibly be created anyway. Python doesn't have a notion of "assignable thing that, when assigned to, will assign to something else" like C's pointers or C++'s references. There's nothing that you could put into the list that would have this behaviour. ChrisA From priisdk at gmail.com Wed Feb 4 16:21:26 2015 From: priisdk at gmail.com (Poul Riis) Date: Wed, 4 Feb 2015 13:21:26 -0800 (PST) Subject: Cairo module In-Reply-To: References: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> Message-ID: <95cd13d2-bd26-4c65-8f42-461d4a5dae6e@googlegroups.com> Could you be a little more specific (giving, for instance, a full working example)? I tried to interchange surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, WIDTH, HEIGHT) with surface = cairo.Win32Surface (cairo.FORMAT_ARGB32, WIDTH, HEIGHT) but that didn't seem to work. Could matplotlib be used to show the image? Poul Riis From rgaddi at technologyhighland.invalid Wed Feb 4 16:28:43 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Wed, 4 Feb 2015 21:28:43 +0000 (UTC) Subject: pymongo and attribute dictionaries References: <436419A8-6CD1-43DA-AC5E-A2E974AD585E@gmail.com> <70CF795C-8494-48CF-A5F8-3D3F8FE3AE95@gmail.com> Message-ID: On Wed, 04 Feb 2015 13:54:22 -0700, Ian Kelly wrote: > I'd prefer map (or itertools.imap in Python 2) over the inline generator > in this case: > > for doc in map(Doc, client.db.radios.find({?_id': {?$regex?: > ?^[ABC]'}})): > pprint(doc) > > Or if you like, a utility function wrapping the same. > > def docs(dicts): > return map(Doc, dicts) Or, if you really wanted to be crazy for d in client.db.radios.find({?_id': {?$regex?: ?^[ABC]'}}): doc = Doc(d) pprint(doc) I mean, I realize linefeeds don't grow on trees and all... -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From steve+comp.lang.python at pearwood.info Wed Feb 4 17:38:26 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 05 Feb 2015 09:38:26 +1100 Subject: pymongo and attribute dictionaries References: Message-ID: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> Travis Griggs wrote: > I really like pymongo. And I really like Python. But one thing my fingers > really get tired of typing is > > someDoc[?_?id?] I've never used pymongo, so can't comment from experience, but surely any library which encourages, if not requires, that you access private data _id doesn't inspire me with confidence. Also, the above is a syntax error, but you probably know that. > This just does not roll of the fingers well. Too many ?reach for modifier > keys? in a row. *One* modifier key in a row is too many? s o m e SHIFT D o c [ ' SHIFT _ i d ' ] You can cut that by 50% by changing your naming convention: somedoc['_id'] While you're at it, you can cut the total number of keystrokes too: doc['_id'] If there are a fixed set of key names that you use repeatedly, you can do this: ID, FE, FI, FO, FUM = '_id', '_fe', '_fi', '_fo', '_fum' # later someDoc[ID] Ah, but that needs the shiftkey... well, I guess that PEP 8 naming conventions aren't compulsory, and so long as you don't need the built-in `id` function, it's okay to shadow it (consenting adults and all that), so: id, fe, fi, fo, fum = '_id', '_fe', '_fi', '_fo', '_fum' # later someDoc[id] > I would rather use > someDoc._id That has exactly the same number of modifier keys as your original example. I'm starting to sense that you don't actually care about the SHIFT key... [...] > The problem I have is not how to do the AttributeDictionary subclass, > there are plenty of those examples. The problem is that the pymongo APIs > already return dictionaries. In a language (Smalltalk, Objective-C, Ruby) > that supports class extensions, that would be my first tool of choice to > solve this problem. I?d just extend Dictionary to behave the way I want > and be done with it. I can?t do that in Python though. I guess I could > make my own module that subclasses the relevant pymongo classes, and do > super() calling implementations of all of the relevant methods, coercing > the return type. That is a maintenance headache though. > > What are my options, if any? HingTFU comes to mind :-) But as "an academic exercise" *nudge nudge, wink wink* there's always the Adaptor design pattern. Import as much or as little of pymongo as you need, wrap it in an adaptor, and use that. There are lots of different ways that you could do this, some more monkey-patchy than others. # pymongo adaptor module (untested) from pymongo import * import functools def wrap(name): """Wrap function called `name` so it returns an AttrDict instead of dict. May the gods have mercy on your soul.""" func = globals()[name] @functools.wraps(func) def inner(*args, **kwargs): result = func(*args, **kwargs) if isinstance(result, dict): result = AttrDict(result) return result globals()[name] = func FUNCTIONS_THAT_RETURN_DICTS = ['this', 'that', 'another'] for name in FUNCTIONS_THAT_RETURN_DICTS: wrap(name) Extending this to use introspection to automatically detect the pymongo functions instead of having to list them manually is left as an exercise. (Hint: import pymongo; for obj in vars(pymongo): ...). Extending this to wrap methods of classes is also left as an exercise. (Hint: don't subclass. Search the ActiveState Python recipes for "automatic delegation" by Alex Martelli.) And now just use the adaptor module instead of the original. -- Steven From cousinstanley at gmail.com Wed Feb 4 17:56:07 2015 From: cousinstanley at gmail.com (Cousin Stanley) Date: Wed, 04 Feb 2015 15:56:07 -0700 Subject: Cairo module References: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> <95cd13d2-bd26-4c65-8f42-461d4a5dae6e@googlegroups.com> Message-ID: > .... > Could matplotlib be used to show the image? You might consider using python-imaging to display the image after writing it from cairo .... import image .... surface.write_to_png ( "x_surface.png" ) img = Image.open( "x_surface.png" ) img.show( command = 'display' ) -- Stanley C. Kitching Human Being Phoenix, Arizona From cousinstanley at gmail.com Wed Feb 4 18:00:17 2015 From: cousinstanley at gmail.com (Cousin Stanley) Date: Wed, 04 Feb 2015 16:00:17 -0700 Subject: Cairo module References: <361dc1c7-a26c-40de-8b60-910f25bdabd9@googlegroups.com> <95cd13d2-bd26-4c65-8f42-461d4a5dae6e@googlegroups.com> Message-ID: > You might consider using python-imaging > to display the image after writing it > from cairo .... > > import image import statement should be .... import Image note uppercase I .... -- Stanley C. Kitching Human Being Phoenix, Arizona From davea at davea.name Wed Feb 4 18:52:30 2015 From: davea at davea.name (Dave Angel) Date: Wed, 04 Feb 2015 18:52:30 -0500 Subject: ANN: unpyc3 - a python bytecode decompiler for Python3 In-Reply-To: <54c98027$0$12982$c3e8da3$5496439d@news.astraweb.com> References: <3dcc2862-bdd6-4a2a-8edb-1a9d51df26b3@googlegroups.com> <54c98027$0$12982$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54D2B0BE.9050805@davea.name> On 01/28/2015 07:34 PM, Steven D'Aprano wrote: > Devin Jeanpierre wrote: > >> On Wed, Jan 28, 2015 at 1:40 PM, Chris Angelico wrote: >>> On Thu, Jan 29, 2015 at 5:47 AM, Chris Kaynor >>> wrote: >>>> I use Google Drive for it for all the stuff I do at home, and use SVN >>>> for all my personal projects, with the SVN depots also in Drive. The >>>> combination works well for me, I can transfer between my desktop and >>>> laptop freely, and have full revision history for debugging issues. >>> >>> I just do everything in git, no need for either Drive or something as >>> old as SVN. Much easier. :) >> >> Git doesn't help if you lose your files in between commits, > > Sure it does? You just lose the changes made since the previous commit, but > that's no different from restoring from backup. The restored file is only > as up to date as the last time a backup was taken. > > >> or if you >> lose the entire directory between pushes. > > Then restore from wherever you are pushing to. > > But as Devin says, any backup strategy that requires the user to make a > backup is untrustworthy. I'm hoping that the next generation of DVCSs will > support continuous commits, the next generation of editors support > continuous saves, and the only time you need interact with the editor > (apart from, you know, actual editing) is to tell it "start a new branch > now". In emacs, bnd the git add, git commit to Ctrl-x - s, and saving also means committing. My backup system uses MD5's to decide which files need backing up, so theoretically it shouldn't cost too much to backup the git archives once daily. It's all still under development, however. (I've been offline for two weeks, developing and running the backup system, and preparing for a complete reinstall of a corrupted system) -- DaveA From sohcahtoa82 at gmail.com Wed Feb 4 19:19:35 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Wed, 4 Feb 2015 16:19:35 -0800 (PST) Subject: ANN: unpyc3 - a python bytecode decompiler for Python3 In-Reply-To: <54c97f06$0$12982$c3e8da3$5496439d@news.astraweb.com> References: <3dcc2862-bdd6-4a2a-8edb-1a9d51df26b3@googlegroups.com> <925339b3-11e9-45b8-a5bb-e0bdf66fbba5@googlegroups.com> <54c97f06$0$12982$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5c35fafd-85ce-44d6-9573-7dc2061b0a10@googlegroups.com> On Wednesday, January 28, 2015 at 4:30:11 PM UTC-8, Steven D'Aprano wrote: > sohcahtoa82 at gmail.com wrote: > > > I recently finished my CS degree, and I had more than one professor say > > that they won't take "My computer crashed and I lost everything!" as an > > excuse for not being able to turn in homework. ? > > How about "My computer crashed and died and now I can't get to Dropbox to > access my files"? If you have access to a web browser, you have access to your Dropbox files. I don't own a printer. If I needed to print something for school, I just printed it from my Dropbox at school. > "My computer got infected by ransomware which encrypted all my data files > and blocks access to Dropbox." Dropbox saves previous versions of files. Just restore them from the last version you had before they got encrypted by the ransomware. This can be done from any browser. > "One of my housemates torrented a Linux tarball, and the MPAA wrongly > identified it as a movie file. Purely on their say-so, my ISP disabled my > account and banned me from the Internet. But all is not lost, if I move 45 > miles away, I can sign up with a different ISP!" Not exactly a likely scenario, and still not a problem unless you're not allowed to access the internet from someone's WiFi. > "Some dude I've never seen before gate-crashed our party and was smoking > pot, and the police raided us and seized my computer and everybody's > phones. My lawyer tells me the raid was illegal and if spend two or three > hundred thousand dollars in legal fees, I'll probably be able to get my > computer back within eight years or so." They can take your computer and it doesn't matter if you've got your files on Dropbox. > "My dog ate my USB stick." > > :-) I never used a USB stick for school work. At this point, I'm probably sounding like a shill for Dropbox, but I'm really not. I imagine Google Drive offers the same features. Access to your files from the web, synchronization of local files among computers with access to it, and the ability to retrieve and restore files from previous versions. From ian.g.kelly at gmail.com Wed Feb 4 19:19:39 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Feb 2015 17:19:39 -0700 Subject: pymongo and attribute dictionaries In-Reply-To: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 4, 2015 at 3:38 PM, Steven D'Aprano wrote: > Travis Griggs wrote: >> This just does not roll of the fingers well. Too many ?reach for modifier >> keys? in a row. > > *One* modifier key in a row is too many? > > s o m e SHIFT D o c [ ' SHIFT _ i d ' ] I think the point was meant to be the number of keys that need to be reached for, not just the number of times Shift needs to be pressed. > Extending this to wrap methods of classes is also left as an exercise. > (Hint: don't subclass. Search the ActiveState Python recipes for "automatic > delegation" by Alex Martelli.) Do you mean this one? http://code.activestate.com/recipes/52295-automatic-delegation-as-an-alternative-to-inherita/ That's based on old-style classes. With new-style classes it fails to delegate dunder methods like __str__. That recipe should be considered obsolete. From torriem at gmail.com Wed Feb 4 19:37:57 2015 From: torriem at gmail.com (Michael Torrie) Date: Wed, 04 Feb 2015 17:37:57 -0700 Subject: ANN: unpyc3 - a python bytecode decompiler for Python3 In-Reply-To: <5c35fafd-85ce-44d6-9573-7dc2061b0a10@googlegroups.com> References: <3dcc2862-bdd6-4a2a-8edb-1a9d51df26b3@googlegroups.com> <925339b3-11e9-45b8-a5bb-e0bdf66fbba5@googlegroups.com> <54c97f06$0$12982$c3e8da3$5496439d@news.astraweb.com> <5c35fafd-85ce-44d6-9573-7dc2061b0a10@googlegroups.com> Message-ID: <54D2BB65.1090006@gmail.com> On 02/04/2015 05:19 PM, sohcahtoa82 at gmail.com wrote: > They can take your computer and it doesn't matter if you've got your files on Dropbox. > >> "My dog ate my USB stick." >> >> :-) > > I never used a USB stick for school work. > > At this point, I'm probably sounding like a shill for Dropbox, but I'm really not. I imagine Google Drive offers the same features. Access to your files from the web, synchronization of local files among computers with access to it, and the ability to retrieve and restore files from previous versions. In my mind, they are all tools. And no one tool should be used and trusted above all others. Anyone that's programming should be using version control, period. But that's not for backup, and backup can and should be used as well as version control. Everything I work on I commit to git regularly because of the utility git gives me. If I end up trying something that doesn't pan out, I can retrace my steps (that's what branches are for). I don't have to dig through two weeks of hourly backups to find out where it was when I started making a change. Backup and git serve two complementary but different purposes. As well as regularly committing code to Git, I run CrashPlan and on a regular schedule (hourly perhaps) it copies all changes, committed or not, and including the git repo itself to the cloud, and also to my other computer, as well as my parents' computer. CrashPlan makes this stuff easy, so there's no reason not have redundancy. As well, I semi-regularly run a manual rsync backup to three different USB hard drives on a rotating backup. Is this overkill? I don't believe so. It requires virtually no work on my part. I don't see any one cloud service as the best product. Why not use them all? Encrypt if you need to, and sync hourly snapshots to google drive, drop box, and any other free cloud service. From steve+comp.lang.python at pearwood.info Wed Feb 4 20:58:22 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 05 Feb 2015 12:58:22 +1100 Subject: pymongo and attribute dictionaries References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d2ce40$0$21718$c3e8da3@news.astraweb.com> Ian Kelly wrote: >> Extending this to wrap methods of classes is also left as an exercise. >> (Hint: don't subclass. Search the ActiveState Python recipes for >> "automatic delegation" by Alex Martelli.) > > Do you mean this one? > > http://code.activestate.com/recipes/52295-automatic-delegation-as-an- alternative-to-inherita/ > > That's based on old-style classes. With new-style classes it fails to > delegate dunder methods like __str__. That recipe should be considered > obsolete. I'm aware of the problem with dunder methods. But that's not an insurmountable problem, if you need your delegation recipe to support dunders (and you may not), it's fiddly and annoying to do so, but not impossible. There are at least two approaches: - manually delegate to the dunders that you care about with hand- written dunder methods: def __str__(self): return type(self.proxied_object).__str__(self.proxied_object) # or for people who prefer simplicity over correctness # return self.proxied_object.__str__() - write a decorator which inspects the class and automatically adds explicit dunders for you. (There's a recipe on ActiveState for that one too.) Regardless of whether you start with Alex's recipe or not, the idea is to make a proxy for the pymongo classes, and delegate to it rather than subclass or re-invent the wheel. That's a basic design pattern, and if dunders are a problem in Python that Ruby or Java doesn't have, oh well, life wasn't meant to be easy. -- Steven From syedk at pacificloud.com Wed Feb 4 22:07:53 2015 From: syedk at pacificloud.com (syed khalid) Date: Wed, 4 Feb 2015 19:07:53 -0800 Subject: Indentation issues with python Message-ID: I downloaded this code and am attempting to run it. I keep getting indentation error. there is a way to handle it with a editor which can recognize the tab or space issue. I have tried different options such as 2 or 3 spaces or tab to no avail. I have encased the error mesage with line 23 between "****************" import sys import azure import socket from azure.servicebus import ( _service_bus_error_handler ) from azure.servicebus.servicebusservice import ( ServiceBusService, ServiceBusSASAuthentication ) from azure.http import ( HTTPRequest, HTTPError ) from azure.http.httpclient import _HTTPClient class EventHubClient(object): def sendMessage(self,body,partition):eventHubHost = "pac-ns.servicebus.windows.net" httpclient = _HTTPClient(service_instance=self) ******************************************************************** File "test1.py", line 23 def sendMessage(self,body,partition): ^ IndentationError: expected an indented block *********************************************************************** sasKeyName = "SendPolicy" sasKeyValue = "erENqf/5wdWCNEbCA9NsDIRqd5MRKdkii07+wezl/NU=" authentication = ServiceBusSASAuthentication(sasKeyName,sasKeyValue) request = HTTPRequest() request.method = "POST" request.host = eventHubHost request.protocol_override = "https" request.path = "/myhub/publishers/" + partition + "/messages?api-version=2014-05 " request.body = body request.headers.append(('Content-Type', 'application/atom+xml;type=entry;charset =utf-8')) authentication.sign_request(request, httpclient) request.headers.append(('Content-Length', str(len(request.body))) -- *Syed Khalid* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Wed Feb 4 22:19:46 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Feb 2015 14:19:46 +1100 Subject: Indentation issues with python References: Message-ID: <851tm5gi3x.fsf@benfinney.id.au> syed khalid writes: > I downloaded this code and am attempting to run it. I keep getting > indentation error. Indentation is crucial information in Python code. If it is lost, don't waste time trying to guess it; instead, get the correct code. How did you download it? You should download the code as a ?plain text? file. -- \ ?I got some new underwear the other day. Well, new to me.? ?Emo | `\ Philips | _o__) | Ben Finney From tjreedy at udel.edu Wed Feb 4 23:37:22 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 04 Feb 2015 23:37:22 -0500 Subject: Indentation issues with python In-Reply-To: References: Message-ID: > class EventHubClient(object): ... > def sendMessage(self,body,partition): ... > ^ > IndentationError: expected an indented block > *********************************************************************** and 'def' is not indented as it must be. This must be covered in the tutorial. -- Terry Jan Reedy From abhi.darkness at gmail.com Wed Feb 4 23:53:48 2015 From: abhi.darkness at gmail.com (Abhiram R) Date: Thu, 5 Feb 2015 10:23:48 +0530 Subject: Usage of some pastebin service proposed Message-ID: Hey guys, I've noticed a lot of people enquiring about syntactic errors and email somewhat butchers the indentation every now and then and the actual error gets buried in this mess. So is it possible to let everyone know that they need to paste their code on some site like pastebin.com and give us the link here so help can be provided better? Just a thought :) ?Cheers Abhiram R -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Feb 5 00:03:38 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Feb 2015 16:03:38 +1100 Subject: Usage of some pastebin service proposed References: Message-ID: <85wq3xeyqd.fsf@benfinney.id.au> Abhiram R writes: > I've noticed a lot of people enquiring about syntactic errors and email > somewhat butchers the indentation every now and then and the actual error > gets buried in this mess. It is almost never email that butchers the indentation. It is the mis-use of HTML in email. The simplest solution is: don't compose email messages in HTML. > So is it possible to let everyone know that they need to paste their > code on some site like pastebin.com and give us the link here so help > can be provided better? > > Just a thought :) No, it's far better to have the code right in the body of the message where it is available to everyone viewing the message at any point in time, regardless of the temporal availability of some arbitrary URL at some third-party service. -- \ ?It is well to remember that the entire universe, with one | `\ trifling exception, is composed of others.? ?John Andrew Holmes | _o__) | Ben Finney From vito.detullio at gmail.com Thu Feb 5 00:36:56 2015 From: vito.detullio at gmail.com (Vito De Tullio) Date: Thu, 05 Feb 2015 06:36:56 +0100 Subject: pymongo and attribute dictionaries References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: >> This just does not roll of the fingers well. Too many ?reach for modifier >> keys? in a row. > > *One* modifier key in a row is too many? > > s o m e SHIFT D o c [ ' SHIFT _ i d ' ] I'm not OP, but as side note... not everyone has "[" as a direct character on the keyboard. I need to press "AltGr" + "?" (and "AltGr" + "+" to get "]"), so I can feel the OP lamenting :) s o m e SHIFT D o c ALTGR ? ' SHIFT - i d ' ALTRG + -- By ZeD From steve+comp.lang.python at pearwood.info Thu Feb 5 00:43:24 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 05 Feb 2015 16:43:24 +1100 Subject: pymongo and attribute dictionaries References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d302fc$0$11096$c3e8da3@news.astraweb.com> Vito De Tullio wrote: > Steven D'Aprano wrote: > >>> This just does not roll of the fingers well. Too many ?reach for >>> modifier keys? in a row. >> >> *One* modifier key in a row is too many? >> >> s o m e SHIFT D o c [ ' SHIFT _ i d ' ] > > I'm not OP, but as side note... not everyone has "[" as a direct character > on the keyboard. I need to press "AltGr" + "?" (and "AltGr" + "+" to get > "]"), so I can feel the OP lamenting :) Point taken. Thank you. -- Steven From nomail at invalid.com Thu Feb 5 03:28:34 2015 From: nomail at invalid.com (ast) Date: Thu, 5 Feb 2015 09:28:34 +0100 Subject: meaning of: line, = In-Reply-To: <54d227ef$0$3292$426a74cc@news.free.fr> References: <54d227ef$0$3292$426a74cc@news.free.fr> Message-ID: <54d329b6$0$3020$426a34cc@news.free.fr> "ast" a ?crit dans le message de news:54d227ef$0$3292$426a74cc at news.free.fr... thanks for the answers From jeanpierreda at gmail.com Thu Feb 5 04:10:09 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Thu, 5 Feb 2015 01:10:09 -0800 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> Message-ID: On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: > On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten <__peter__ at web.de> wrote: >> Another alternative is to put a list literal on the lefthand side: >> >>>>> def f(): yield 42 >> >> ... >>>>> [result] = f() >>>>> result >> 42 > > Huh, was not aware of that alternate syntax. Nor are most people. Nor is Python, in some places -- it seems like people forgot about it when writing some bits of the grammar. I'd suggest not using it. >> (If you're worried: neither the list nor the tuple will be created; the >> bytecode is identical in both cases) > > It can't possibly be created anyway. Python doesn't have a notion of > "assignable thing that, when assigned to, will assign to something > else" like C's pointers or C++'s references. There's nothing that you > could put into the list that would have this behaviour. C pointers don't do that either. It's really just references. (C pointers aren't any more action-at-a-distance than Python attributes.) Anyway, it could create a new list in Python, because Python can do whatever it wants. But it doesn't, because as you say, that wouldn't do anything. -- Devin From steve+comp.lang.python at pearwood.info Thu Feb 5 04:40:53 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 05 Feb 2015 20:40:53 +1100 Subject: meaning of: line, = References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> Message-ID: <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Devin Jeanpierre wrote: > On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: >> On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten <__peter__ at web.de> wrote: >>> Another alternative is to put a list literal on the lefthand side: >>> >>>>>> def f(): yield 42 >>> >>> ... >>>>>> [result] = f() >>>>>> result >>> 42 >> >> Huh, was not aware of that alternate syntax. > > Nor are most people. Nor is Python, in some places -- it seems like > people forgot about it when writing some bits of the grammar. Got an example where you can use a,b but not [a,b] or (a,b)? -- Steven From denismfmcmahon at gmail.com Thu Feb 5 06:46:05 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 5 Feb 2015 11:46:05 +0000 (UTC) Subject: Indentation issues with python References: Message-ID: On Wed, 04 Feb 2015 19:07:53 -0800, syed khalid wrote: > I downloaded this code and am attempting to run it. I keep getting > indentation error. > class EventHubClient(object): > def sendMessage(self,body,partition):eventHubHost = > "pac-ns.servicebus.windows.net" > httpclient = _HTTPClient(service_instance=self) > def sendMessage(self,body,partition): > ^ > IndentationError: expected an indented block > *********************************************************************** > sasKeyName = "SendPolicy" > sasKeyValue = "erENqf/5wdWCNEbCA9NsDIRqd5MRKdkii07+wezl/NU=" class starts a class definition. def starts a function (or method) definition. The parser expects these definitions to be followed by one or more indented lines being the code of the class or function (method). The following lines down to either a return or the next function definition line should probably all be indented. eg: class EventHubClient(object): def sendMessage(self,body,partition): eventHubHost = "pac-ns.servicebus.windows.net" httpclient = _HTTPClient(service_instance=self) sasKeyName = "SendPolicy" sasKeyValue = "erENqf/5wdWCNEbCA9NsDIRqd5MRKdkii07+wezl/NU=" ...... more indented code should probably follow Getting the indentation correct is absolutely critical in Python. -- Denis McMahon, denismfmcmahon at gmail.com From as at sci.fi Thu Feb 5 08:27:13 2015 From: as at sci.fi (Anssi Saari) Date: Thu, 05 Feb 2015 15:27:13 +0200 Subject: pymongo and attribute dictionaries References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> <54d302fc$0$11096$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano writes: > Vito De Tullio wrote: > >> Steven D'Aprano wrote: >> >>>> This just does not roll of the fingers well. Too many ?reach for >>>> modifier keys? in a row. >>> >>> *One* modifier key in a row is too many? >>> >>> s o m e SHIFT D o c [ ' SHIFT _ i d ' ] >> >> I'm not OP, but as side note... not everyone has "[" as a direct character >> on the keyboard. I need to press "AltGr" + "?" (and "AltGr" + "+" to get >> "]"), so I can feel the OP lamenting :) > > Point taken. Thank you. What I find surprising is that so many people cling so hard to their localized keyboard layouts. I think none of those were created by engineers and should be avoided by technical people. Or, in fact, everyone. Even Microsoft seems to understand this and so Windows installs the US English layout by default as an alternative. From skip.montanaro at gmail.com Thu Feb 5 09:40:47 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 5 Feb 2015 08:40:47 -0600 Subject: Usage of some pastebin service proposed In-Reply-To: References: Message-ID: On Wed, Feb 4, 2015 at 10:53 PM, Abhiram R wrote: > So is it possible to let everyone know that they need to paste their code on > some site like pastebin.com and give us the link here so help can be > provided better? Better would be to attach small code snippets. I agree with Ben about the HTML mail thing. If you insist on using that (sometimes I get into non-plain-text mode in Gmail and forget to turn it off), then use whatever facility your mailer has for quoting text, and set the font to fixed width. try: > opts, args = getopt.getopt(args, "o:i:f:F:DHnh") > except getopt.GetoptError, msg: > usage(msg) > return 1 old-school-argument-parsing-ly, y'rs, Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.g.kelly at gmail.com Thu Feb 5 11:08:30 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 5 Feb 2015 09:08:30 -0700 Subject: meaning of: line, = In-Reply-To: <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 5, 2015 at 2:40 AM, Steven D'Aprano wrote: > Devin Jeanpierre wrote: > >> On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: >>> On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten <__peter__ at web.de> wrote: >>>> Another alternative is to put a list literal on the lefthand side: >>>> >>>>>>> def f(): yield 42 >>>> >>>> ... >>>>>>> [result] = f() >>>>>>> result >>>> 42 >>> >>> Huh, was not aware of that alternate syntax. >> >> Nor are most people. Nor is Python, in some places -- it seems like >> people forgot about it when writing some bits of the grammar. > > Got an example where you can use a,b but not [a,b] or (a,b)? >>> def f(a, (b, c)): ... print a, b, c ... >>> f(3, [4, 5]) 3 4 5 >>> def g(a, [b, c]): File "", line 1 def g(a, [b, c]): ^ SyntaxError: invalid syntax Although to be fair, the first syntax there is no longer valid either in Python 3. From p.f.moore at gmail.com Thu Feb 5 11:20:41 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 5 Feb 2015 08:20:41 -0800 (PST) Subject: Monte Carlo probability calculation in Python Message-ID: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> I'm interested in prototyping a Monte Carlo type simulation algorithm in Python. The background is that a friend has written a similar program in C++, and I'm interested in seeing if I can achieve something comparable in a much better language :-) The basic job of the program will be to simulate games of chance - so we'll have random inputs (die rolls, card draws, etc) and repeatedly simulate calculating a "result". Based on the results of the simulation, the idea is to estimate the probability of a given result. So, to give a very specific example: import random def die(n, sides=6): total = sum(random.randint(1, sides) for i in range(n)) return total def simulate(n, test): "Run the simulation N times, returning the probability that TEST is true" successes = 0 for i in range(n): if test(): successes = successes + 1 return successes/n def check_3d6_gt_15(): return die(3) > 15 if __name__ == '__main__': print(simulate(100000, check_3d6_gt_15)) Obviously, this is going to run ridiculously slowly as the number of simulations or the complexity of the calculation increases, but this gives the idea. My immediate instinct is that somewhere in the scipy stack, there will be a module that does this sort of thing efficiently, but I don't really know where to look - my understanding of the maths involved is very much at the naive level above, so I'm not sure what terms I should be searching for, or how to frame a query. Can anyone give me some pointers as to where I should go to find out more about this sort of task? Either more general theory (that would help me ask the right questions!) or specific packages or techniques I should be using in Python/numpy would be fantastic. Any help would be gratefully accepted - surely nobody wants to see Python beaten by a C++ program??? :-) Thanks, Paul From python.list at tim.thechases.com Thu Feb 5 11:21:12 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 5 Feb 2015 10:21:12 -0600 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20150205102112.521a82aa@bigbox.christie.dr> On 2015-02-05 09:08, Ian Kelly wrote: > > Got an example where you can use a,b but not [a,b] or (a,b)? > > >>> def f(a, (b, c)): > ... print a, b, c > ... Interesting. I knew that at one point you could do this with lambdas but never thought to do it with regular functions. There are times this would have been useful, but since it appears to have gone away in Py3, I guess I won't adopt it. -tkc From joel.goldstick at gmail.com Thu Feb 5 11:27:51 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 5 Feb 2015 11:27:51 -0500 Subject: Monte Carlo probability calculation in Python In-Reply-To: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> Message-ID: On Thu, Feb 5, 2015 at 11:20 AM, Paul Moore wrote: > I'm interested in prototyping a Monte Carlo type simulation algorithm in > Python. The background is that a friend has written a similar program in > C++, and I'm interested in seeing if I can achieve something comparable in > a much better language :-) > > The basic job of the program will be to simulate games of chance - so > we'll have random inputs (die rolls, card draws, etc) and repeatedly > simulate calculating a "result". Based on the results of the simulation, > the idea is to estimate the probability of a given result. > > So, to give a very specific example: > > import random > > def die(n, sides=6): > total = sum(random.randint(1, sides) for i in range(n)) > return total > > def simulate(n, test): > "Run the simulation N times, returning the probability that TEST is > true" > successes = 0 > for i in range(n): > if test(): > successes = successes + 1 > return successes/n > > def check_3d6_gt_15(): > return die(3) > 15 > > if __name__ == '__main__': > print(simulate(100000, check_3d6_gt_15)) > > Obviously, this is going to run ridiculously slowly as the number of > simulations or the complexity of the calculation increases, but this gives > the idea. > > My immediate instinct is that somewhere in the scipy stack, there will be > a module that does this sort of thing efficiently, but I don't really know > where to look - my understanding of the maths involved is very much at the > naive level above, so I'm not sure what terms I should be searching for, or > how to frame a query. > > Can anyone give me some pointers as to where I should go to find out more > about this sort of task? Either more general theory (that would help me ask > the right questions!) or specific packages or techniques I should be using > in Python/numpy would be fantastic. > > Any help would be gratefully accepted - surely nobody wants to see Python > beaten by a C++ program??? :-) > > Thanks, > Paul > -- > https://mail.python.org/mailman/listinfo/python-list > have you googled "python monte carlo"? -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Feb 5 11:33:47 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 5 Feb 2015 08:33:47 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> Message-ID: <63f603bc-431c-45e6-a496-0a28132f99d6@googlegroups.com> On Thursday, 5 February 2015 16:28:07 UTC, Joel Goldstick wrote: > have you googled "python monte carlo"? Yes. And a number of other variations. None gave anything that seemed to relate. It's quite likely though that I'm simply not understanding how things like pymc (which came up in the searches) might help me, or how to convert my problem into a Monte Carlo integration problem (another topic that came up a lot, for example). So if there are specific links from such a search that match well to the problem as I described it above, I'd be really grateful for pointers. Paul From rustompmody at gmail.com Thu Feb 5 11:45:15 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 5 Feb 2015 08:45:15 -0800 (PST) Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> On Thursday, February 5, 2015 at 9:39:27 PM UTC+5:30, Ian wrote: > On Thu, Feb 5, 2015 at 2:40 AM, Steven D'Aprano wrote: > > Devin Jeanpierre wrote: > > > >> On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: > >>> On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten wrote: > >>>> Another alternative is to put a list literal on the lefthand side: > >>>> > >>>>>>> def f(): yield 42 > >>>> > >>>> ... > >>>>>>> [result] = f() > >>>>>>> result > >>>> 42 > >>> > >>> Huh, was not aware of that alternate syntax. > >> > >> Nor are most people. Nor is Python, in some places -- it seems like > >> people forgot about it when writing some bits of the grammar. > > > > Got an example where you can use a,b but not [a,b] or (a,b)? > > >>> def f(a, (b, c)): > ... print a, b, c What the hell is that?! First I am hearing/seeing it. Whats it called? From rgaddi at technologyhighland.invalid Thu Feb 5 11:56:19 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Thu, 5 Feb 2015 16:56:19 +0000 (UTC) Subject: Monte Carlo probability calculation in Python References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> Message-ID: On Thu, 05 Feb 2015 08:20:41 -0800, Paul Moore wrote: > I'm interested in prototyping a Monte Carlo type simulation algorithm in > Python. The background is that a friend has written a similar program in > C++, and I'm interested in seeing if I can achieve something comparable > in a much better language :-) > > The basic job of the program will be to simulate games of chance - so > we'll have random inputs (die rolls, card draws, etc) and repeatedly > simulate calculating a "result". Based on the results of the simulation, > the idea is to estimate the probability of a given result. > > So, to give a very specific example: > > import random > > def die(n, sides=6): > total = sum(random.randint(1, sides) for i in range(n)) > return total > > def simulate(n, test): > "Run the simulation N times, returning the probability that TEST is > true" > successes = 0 for i in range(n): > if test(): > successes = successes + 1 > return successes/n > > def check_3d6_gt_15(): > return die(3) > 15 > > if __name__ == '__main__': > print(simulate(100000, check_3d6_gt_15)) > > Obviously, this is going to run ridiculously slowly as the number of > simulations or the complexity of the calculation increases, but this > gives the idea. > > My immediate instinct is that somewhere in the scipy stack, there will > be a module that does this sort of thing efficiently, but I don't really > know where to look - my understanding of the maths involved is very much > at the naive level above, so I'm not sure what terms I should be > searching for, or how to frame a query. > > Can anyone give me some pointers as to where I should go to find out > more about this sort of task? Either more general theory (that would > help me ask the right questions!) or specific packages or techniques I > should be using in Python/numpy would be fantastic. > > Any help would be gratefully accepted - surely nobody wants to see > Python beaten by a C++ program??? :-) > > Thanks, > Paul You don't need the whole scipy stack, numpy will let you do everything you want. The trick to working in numpy is to parallelize your problem; you don't do a thing a thousand times; you do it on a thousand-length array. For example: def dice(throws, per, sides=6): """Return an array throws long of rolls of (per)d(sides).""" all_dice = np.random.randint(1, sides+1, size=(throws, per)) return all_dice.sum(axis=1) -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From skip.montanaro at gmail.com Thu Feb 5 11:56:51 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 5 Feb 2015 10:56:51 -0600 Subject: meaning of: line, = In-Reply-To: <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> Message-ID: Tuple packing. No longer supported in Python 3, but in available in Python <= 2. Skip On Thu, Feb 5, 2015 at 10:45 AM, Rustom Mody wrote: > On Thursday, February 5, 2015 at 9:39:27 PM UTC+5:30, Ian wrote: >> On Thu, Feb 5, 2015 at 2:40 AM, Steven D'Aprano wrote: >> > Devin Jeanpierre wrote: >> > >> >> On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: >> >>> On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten wrote: >> >>>> Another alternative is to put a list literal on the lefthand side: >> >>>> >> >>>>>>> def f(): yield 42 >> >>>> >> >>>> ... >> >>>>>>> [result] = f() >> >>>>>>> result >> >>>> 42 >> >>> >> >>> Huh, was not aware of that alternate syntax. >> >> >> >> Nor are most people. Nor is Python, in some places -- it seems like >> >> people forgot about it when writing some bits of the grammar. >> > >> > Got an example where you can use a,b but not [a,b] or (a,b)? >> >> >>> def f(a, (b, c)): >> ... print a, b, c > > What the hell is that?! > First I am hearing/seeing it. > Whats it called? > -- > https://mail.python.org/mailman/listinfo/python-list From rustompmody at gmail.com Thu Feb 5 11:59:54 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 5 Feb 2015 08:59:54 -0800 (PST) Subject: meaning of: line, = In-Reply-To: <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> Message-ID: <98777b0d-8f6c-4e2d-965f-0ef875e962b0@googlegroups.com> On Thursday, February 5, 2015 at 10:15:29 PM UTC+5:30, Rustom Mody wrote: > On Thursday, February 5, 2015 at 9:39:27 PM UTC+5:30, Ian wrote: > > On Thu, Feb 5, 2015 at 2:40 AM, Steven D'Aprano wrote: > > > Devin Jeanpierre wrote: > > > > > >> On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: > > >>> On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten wrote: > > >>>> Another alternative is to put a list literal on the lefthand side: > > >>>> > > >>>>>>> def f(): yield 42 > > >>>> > > >>>> ... > > >>>>>>> [result] = f() > > >>>>>>> result > > >>>> 42 > > >>> > > >>> Huh, was not aware of that alternate syntax. > > >> > > >> Nor are most people. Nor is Python, in some places -- it seems like > > >> people forgot about it when writing some bits of the grammar. > > > > > > Got an example where you can use a,b but not [a,b] or (a,b)? > > > > >>> def f(a, (b, c)): > > ... print a, b, c > > What the hell is that?! > First I am hearing/seeing it. > Whats it called? The reason I ask: I sorely miss haskell's pattern matching in python. It goes some way: >>> ((x,y),z) = ((1,2),3) >>> x,y,z (1, 2, 3) But not as far as I would like: >>> ((x,y),3) = ((1,2),3) File "", line 1 SyntaxError: can't assign to literal >>> [Haskell] Prelude> let (x, (y, (42, z, "Hello"))) = (1, (2, (42, 3, "Hello"))) Prelude> (x,y,z) (1,2,3) From python.list at tim.thechases.com Thu Feb 5 12:00:46 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 5 Feb 2015 11:00:46 -0600 Subject: meaning of: line, = In-Reply-To: <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> Message-ID: <20150205110046.7f643d7f@bigbox.christie.dr> On 2015-02-05 08:45, Rustom Mody wrote: > > >>> def f(a, (b, c)): > > ... print a, b, c > > What the hell is that?! > First I am hearing/seeing it. > Whats it called? "tuple parameter unpacking", removed in Py3 https://www.python.org/dev/peps/pep-3113/ -tkc From johnnybureddu at gmx.com Thu Feb 5 14:15:02 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Thu, 5 Feb 2015 11:15:02 -0800 (PST) Subject: -- redacted -- Message-ID: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> -- redacted -- From p.f.moore at gmail.com Thu Feb 5 14:25:42 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 5 Feb 2015 11:25:42 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> Message-ID: <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> On Thursday, 5 February 2015 16:57:07 UTC, Rob Gaddi wrote: > You don't need the whole scipy stack, numpy will let you do everything > you want. The trick to working in numpy is to parallelize your problem; > you don't do a thing a thousand times; you do it on a thousand-length > array. For example: > > def dice(throws, per, sides=6): > """Return an array throws long of rolls of (per)d(sides).""" > all_dice = np.random.randint(1, sides+1, size=(throws, per)) > return all_dice.sum(axis=1) Thanks, that's a help. I see the principle, but a couple of questions. With bigger problems (deal 52 cards into bridge hands a million times, for example) would memory become an issue? Also, how do you handle things that don't fit into the built-in numpy operations? (For example, Monopoly - roll 2 dice and take the sum, unless you roll a double, in which case reroll, but if you roll 3 doubles you fail - return NaN in that case). As examples of a few more problems I'd use this for: 1. Roll 4 dice, and sum the largest 3. 2. Roll 9 dice, count how many different numbers you get. 3. Deal 4 hands of 13 cards from a full deck, determine the length of the longest suit anyone has. 4. Monopoly rolls (see above, 2 dice but reroll doubles up to 3 times) All of these are reasonably easy to write as Python functions, but it becomes progressively harder (for a non-numpy expert) to know how to convert them into parallelizable numpy primitives. I'll have a play with the approach you suggest though, and see how it handles some of the more complex problems my friend was showing off with :-) (his worst example was a horrendous expression to score Cribbage hands, that if I read his note correctly took about a minute to simulate 10,000,000 deals) Paul From jessemuzziddu at mail.com Thu Feb 5 14:47:54 2015 From: jessemuzziddu at mail.com (CHE SIA REVOLUCIOOON!!!) Date: Thu, 5 Feb 2015 11:47:54 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <68319142-cff1-4bbd-a73e-eb5790f721fc@googlegroups.com> -- redacted -- From ole-usenet-spam at gmx.net Thu Feb 5 15:22:33 2015 From: ole-usenet-spam at gmx.net (=?utf-8?B?0J5s0LUg0IV0ctC10ZbRgWjQtXI=?=) Date: Thu, 05 Feb 2015 21:22:33 +0100 Subject: multiprocessing.Queue() and missing sem_open Message-ID: <877fvwaz1y.fsf@news.ole.ath.cx> Hi, I am just trying to prepare a package (astropy) for (Debian) Hurd. This os lacks a sem_open() implementation. When I now try: import multiprocessing q = multiprocessing.Queue() I get an ImportError with Python 2.7, but an AttributeError with Python 3.4. In the documentation of multiprocessing.Queue() I couldn't find any hint that it would throw this exception. I am now curious, that 1. this behaviour is not documented 2. it changed at some point, without documentation. Why does it not return a NotImplementedError (this is what I would expect if a function is not implemented by the OS)? Can I be sure that the following works also in future? try q = multiprocessing.Queue() except (ImportError, AttributeError) # handle the case of missing sem_open Or what is the correct way to catch a not working Queue caused by a missing sem_open() implementation? Best regards Ole From rgaddi at technologyhighland.invalid Thu Feb 5 15:56:14 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Thu, 5 Feb 2015 20:56:14 +0000 (UTC) Subject: Monte Carlo probability calculation in Python References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> Message-ID: On Thu, 05 Feb 2015 11:25:42 -0800, Paul Moore wrote: > On Thursday, 5 February 2015 16:57:07 UTC, Rob Gaddi wrote: >> You don't need the whole scipy stack, numpy will let you do everything >> you want. The trick to working in numpy is to parallelize your >> problem; >> you don't do a thing a thousand times; you do it on a thousand-length >> array. For example: >> >> def dice(throws, per, sides=6): >> """Return an array throws long of rolls of (per)d(sides).""" >> all_dice = np.random.randint(1, sides+1, size=(throws, per)) >> return all_dice.sum(axis=1) > > Thanks, that's a help. I see the principle, but a couple of questions. > With bigger problems (deal 52 cards into bridge hands a million times, > for example) would memory become an issue? Also, how do you handle > things that don't fit into the built-in numpy operations? (For example, > Monopoly - roll 2 dice and take the sum, unless you roll a double, in > which case reroll, but if you roll 3 doubles you fail - return NaN in > that case). > > As examples of a few more problems I'd use this for: > > 1. Roll 4 dice, and sum the largest 3. > 2. Roll 9 dice, count how many different numbers you get. > 3. Deal 4 hands of 13 cards from a full deck, determine the length of > the longest suit anyone has. > 4. Monopoly rolls (see above, 2 dice but reroll doubles up to 3 times) > > All of these are reasonably easy to write as Python functions, but it > becomes progressively harder (for a non-numpy expert) to know how to > convert them into parallelizable numpy primitives. > > I'll have a play with the approach you suggest though, and see how it > handles some of the more complex problems my friend was showing off with > :-) (his worst example was a horrendous expression to score Cribbage > hands, that if I read his note correctly took about a minute to simulate > 10,000,000 deals) > > Paul A lot of your examples can be dealt by the numpy fancy indexing tricks, which is really the heart of numpy. A comparison on an array of numbers gives an array of bools, which can then be used to index the original array. The Monopoly example, for instance. all_dice = np.random.randint(1, 7, size=(throws, 2)) doubles = all_dice[all_dice[:,0] == all_dice[:,1]] doubles is now a writeable view into the original all_dice array, so you could reroll them by saying something like. doubles[:,:] = np.random.randint(1, 7, size=doubles.shape) Memory will eventually become a concern, but in that case you could always run it in parts and aggregate the parts. I wouldn't sweat it until you get into 100s of millions of tries. Even then, you could make some improvements by forcing the dtypes to be uint8, though there may be a performance penalty. Numpy is fantastically powerful once you can wrap your head around it; I use it for a ton of things I used to use Matlab for. The tutorial at http://wiki.scipy.org/Tentative_NumPy_Tutorial will go into lots of depth for you. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ian.g.kelly at gmail.com Thu Feb 5 16:00:20 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 5 Feb 2015 14:00:20 -0700 Subject: Monte Carlo probability calculation in Python In-Reply-To: <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> Message-ID: On Thu, Feb 5, 2015 at 12:25 PM, Paul Moore wrote: > On Thursday, 5 February 2015 16:57:07 UTC, Rob Gaddi wrote: >> You don't need the whole scipy stack, numpy will let you do everything >> you want. The trick to working in numpy is to parallelize your problem; >> you don't do a thing a thousand times; you do it on a thousand-length >> array. For example: >> >> def dice(throws, per, sides=6): >> """Return an array throws long of rolls of (per)d(sides).""" >> all_dice = np.random.randint(1, sides+1, size=(throws, per)) >> return all_dice.sum(axis=1) > > Thanks, that's a help. I see the principle, but a couple of questions. With bigger problems (deal 52 cards into bridge hands a million times, for example) would memory become an issue? At the point memory becomes an issue you can partially roll it back into a loop. For example, deal the bridge hands 10000 times in a loop of 100. > Also, how do you handle things that don't fit into the built-in numpy operations? (For example, Monopoly - roll 2 dice and take the sum, unless you roll a double, in which case reroll, but if you roll 3 doubles you fail - return NaN in that case). Building on Rob's example: def monopoly(throws, per=2, rerolls=3, sides=6): all_dice = np.random.randint(1, sides+1, size=(throws, rerolls, per)) doubles = all_dice[...,0] == all_dice[...,1] three_doubles = doubles[:,0] & doubles[:,1] & doubles[:,2] return all_dice.sum(axis=2), doubles, three_doubles This returns a (throws x rerolls) array of the sum of each roll, a (throws x rerolls) array of booleans indicating whether the roll was a double or not, and a throws-long array of booleans indicating whether three doubles were rolled. From ian.g.kelly at gmail.com Thu Feb 5 16:11:38 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 5 Feb 2015 14:11:38 -0700 Subject: pymongo and attribute dictionaries In-Reply-To: References: <54d29f63$0$12986$c3e8da3$5496439d@news.astraweb.com> <54d302fc$0$11096$c3e8da3@news.astraweb.com> Message-ID: On Thu, Feb 5, 2015 at 6:27 AM, Anssi Saari wrote: > What I find surprising is that so many people cling so hard to their > localized keyboard layouts. I think none of those were created by > engineers and should be avoided by technical people. Or, in fact, > everyone. Even Microsoft seems to understand this and so Windows > installs the US English layout by default as an alternative. Speaking as a user of both Dvorak and Qwerty, switching between layouts frequently can be confusing. In particular the pain points are the places where the two layouts are similar. I haven't done this so much recently, but in the past I have often been typing away in one layout, then come to an A or an M (which happen to be in the same position in both layouts) at which point my brain would inadvertently "switch" and I would suddenly find myself typing in the wrong layout. I would expect that a localized layout with more similarity than what there is between Dvorak and Qwerty would cause even more of these types of problems. From rosuav at gmail.com Thu Feb 5 17:29:05 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Feb 2015 09:29:05 +1100 Subject: multiprocessing.Queue() and missing sem_open In-Reply-To: <877fvwaz1y.fsf@news.ole.ath.cx> References: <877fvwaz1y.fsf@news.ole.ath.cx> Message-ID: On Fri, Feb 6, 2015 at 7:22 AM, ?l? ?tr???h?r wrote: > I am just trying to prepare a package (astropy) for (Debian) Hurd. This > os lacks a sem_open() implementation. When I now try: > > import multiprocessing > q = multiprocessing.Queue() > > I get an ImportError with Python 2.7, but an AttributeError with Python > 3.4. In the documentation of multiprocessing.Queue() I couldn't find any > hint that it would throw this exception. Neither of those errors is being consciously thrown by the Queue class; the latter means that multiprocessing exists but it has no Queue in it, and the former means that the entire multiprocessing module is absent. I'm guessing you built Python from source? If so, you should have had a report at the bottom of the build output stating which of the optional packages weren't built, which on 2.7 will have included multiprocessing. You might be able to install some dependency and get that to build, though I don't know what it would be, given that the 3.4 build managed to find it. But this is the exact way that Python will normally signal that something isn't available. If your platform doesn't have, say, os.O_BINARY, because that flag has no meaning for you, then you don't get NotImplementedError - you simply get AttributeError. > Can I be sure that the following works also in future? > > try > q = multiprocessing.Queue() > except (ImportError, AttributeError) > # handle the case of missing sem_open > > Or what is the correct way to catch a not working Queue caused by a > missing sem_open() implementation? The ImportError will come from the import statement, the AttributeError will come from the attribute lookup - the above code can't[1] bomb out with ImportError. So a more correct way would be something like: try: import multiprocessing multiprocessing.Queue except (ImportError, AttributeError): # handle the absence Or alternatively, don't even bother to try/except the import. If your code requires multiprocessing.Queue (as opposed to just queue.Queue), chances are you can't cope with the absence of the module. ChrisA [1] Yeah yeah, anything can do anything. You know what I mean. From p.f.moore at gmail.com Thu Feb 5 18:02:24 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 5 Feb 2015 15:02:24 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> Message-ID: On Thursday, 5 February 2015 21:01:21 UTC, Ian wrote: > Building on Rob's example: > > def monopoly(throws, per=2, rerolls=3, sides=6): > all_dice = np.random.randint(1, sides+1, size=(throws, rerolls, per)) > doubles = all_dice[...,0] == all_dice[...,1] > three_doubles = doubles[:,0] & doubles[:,1] & doubles[:,2] > return all_dice.sum(axis=2), doubles, three_doubles > > This returns a (throws x rerolls) array of the sum of each roll, a > (throws x rerolls) array of booleans indicating whether the roll was a > double or not, and a throws-long array of booleans indicating whether > three doubles were rolled. Nice! Thanks both of you. I start to see how to think about these things now. I'll play around with some of the other examples I've got, and see how far I get. P From greg.ewing at canterbury.ac.nz Thu Feb 5 18:30:39 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Feb 2015 12:30:39 +1300 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> Message-ID: Devin Jeanpierre wrote: > On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: > >>On Thu, Feb 5, 2015 at 4:36 AM, Peter Otten <__peter__ at web.de> wrote: >> >>>>>>>[result] = f() >>>>>>result >> >>Huh, was not aware of that alternate syntax. > > Nor are most people. Nor is Python, in some places -- it seems like > people forgot about it when writing some bits of the grammar. If I remember correctly, it's left over from long ago when you had to use tuple syntax to unpack tuples and list syntax to unpack lists (and you couldn't unpack anything else). When the iterator protocol was introduced, the two became equivalent. -- Greg From jeanpierreda at gmail.com Thu Feb 5 20:12:17 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Thu, 5 Feb 2015 17:12:17 -0800 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 5, 2015 at 8:08 AM, Ian Kelly wrote: > On Thu, Feb 5, 2015 at 2:40 AM, Steven D'Aprano > wrote: >> Devin Jeanpierre wrote: >>> On Wed, Feb 4, 2015 at 1:18 PM, Chris Angelico wrote: >>>>>>>> [result] = f() >>>>>>>> result >>>>> 42 >>>> >>>> Huh, was not aware of that alternate syntax. >>> >>> Nor are most people. Nor is Python, in some places -- it seems like >>> people forgot about it when writing some bits of the grammar. >> >> Got an example where you can use a,b but not [a,b] or (a,b)? > >>>> def f(a, (b, c)): > ... print a, b, c > ... >>>> f(3, [4, 5]) > 3 4 5 >>>> def g(a, [b, c]): > File "", line 1 > def g(a, [b, c]): > ^ > SyntaxError: invalid syntax > > Although to be fair, the first syntax there is no longer valid either > in Python 3. As Ian rightly understood, I was referring to differences between "[a, b, ...]" and "(a, b, ...)". Here's another example, one that still exists in Python 3: >>> [] = '' >>> () = '' File "", line 1 SyntaxError: can't assign to () The syntax explicitly blacklists (), but forgets to blacklist []. -- Devin From rosuav at gmail.com Thu Feb 5 20:17:31 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Feb 2015 12:17:31 +1100 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Feb 6, 2015 at 12:12 PM, Devin Jeanpierre wrote: > Here's another example, one that still exists in Python 3: > >>>> [] = '' >>>> () = '' > File "", line 1 > SyntaxError: can't assign to () > > The syntax explicitly blacklists (), but forgets to blacklist []. So... this is actually a really really obscure little feature. [] = x # is equivalent to try: next(iter(x)) except StopIteration: pass else: raise ValueError("too many values to unpack (expected 0)") It's a way of asserting that an iterator is exhausted! Perfect code snippet for your next International Obfuscated Python Code Contest entry. ChrisA From greg.ewing at canterbury.ac.nz Thu Feb 5 23:52:11 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Feb 2015 17:52:11 +1300 Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: Chris Angelico wrote: > [] = x > # is equivalent to > try: next(iter(x)) > except StopIteration: pass > else: raise ValueError("too many values to unpack (expected 0)") > > It's a way of asserting that an iterator is exhausted! But why disallow using () for the same thing? This is a blatant case of outright list-ism! Tuples are being unfairly discriminated against! -- Greg From syedk at pacificloud.com Fri Feb 6 00:26:20 2015 From: syedk at pacificloud.com (syed khalid) Date: Thu, 5 Feb 2015 21:26:20 -0800 Subject: Azure event hub network access Message-ID: I am getting http error 404. I am able to access the site via telnet which eliminates network issues. Here is the code and subsequent errors user/bin/python import sys import azure import socket from azure.servicebus import ( _service_bus_error_handler ) from azure.servicebus.servicebusservice import ( ServiceBusService, ServiceBusSASAuthentication ) from azure.http import ( HTTPRequest, HTTPError ) from azure.http.httpclient import _HTTPClient class EventHubClient(object): def sendMessage(self,body,partition): eventHubHost = "pac-ns.servicebus.windows.net" httpclient = _HTTPClient(service_instance=self) sasKeyName = "pac-pl" sasKeyValue = "IhkEepQPLfSy9jo6H2Yxxxxxxxxxxxxxxxxxxxx=" authentication = ServiceBusSASAuthentication(sasKeyName,sasKeyValue) request = HTTPRequest() request.method = "POST" request.host = eventHubHost request.protocol_override = "https" # request.path = "/myhub/publishers/" + partition + "/messages?api-version=20 14-05" request.body = body request.headers.append(('Content-Type', 'application/atom+xml;type=entry;cha rset=utf-8')) authentication.sign_request(request, httpclient) request.headers.append(('Content-Length', str(len(request.body)))) status = 0 try: resp = httpclient.perform_request(request) status = resp.status except HTTPError as ex: status = ex.status return status class EventDataParser(object): def getMessage(self,payload,sensorId): host = socket.gethostname() body = "{ \"DeviceId\" : \"" + host + "\",\"SensorData\": [ " msgs = payload.split(",") first = True for msg in msgs: # print msg sensorType = msg.split(":")[0] sensorValue = msg.split(":")[1] if first == True: first = False else: body += "," body += "{ \"SensorId\" : \"" + sensorId + "\", \"SensorType\" : \"" + sen sorType + "\", \"SensorValue\" : " + sensorValue + " }" body += "]}" return body hubClient = EventHubClient() parser = EventDataParser() hostname = socket.gethostname() sensor = sys.argv[2] body = parser.getMessage(sys.argv[1],sensor) hubStatus = hubClient.sendMessage(body,hostname) # return the HTTP status to the caller print hubStatus print hostname print sensor ~/IOT/AZURE$ python send.py temperature:22,humidity:20 deviceid 404 ubuntu deviceid { "DeviceId" : "ubuntu","SensorData": [ { "SensorId" : "deviceid", "SensorType" : "temperature", "SensorValue" : 22 },{ "SensorId" : "deviceid", "SensorType" : "humidity", "SensorValue" : 20 }]} -- *Syed Khalid* -------------- next part -------------- An HTML attachment was scrubbed... URL: From Steve.Dower at microsoft.com Fri Feb 6 00:45:37 2015 From: Steve.Dower at microsoft.com (Steve Dower) Date: Fri, 6 Feb 2015 05:45:37 +0000 Subject: [Python-Dev] Azure event hub network access In-Reply-To: References: Message-ID: This would be much better posted on the github page for the project. I don't have the URL handy, but if you search github for "Python Azure SDK" you'll find it. Cheers, Steve Sent from my Windows Phone ________________________________ From: syed khalid Sent: ?2/?5/?2015 21:27 To: python-dev at python.org Cc: python-list at python.org Subject: [Python-Dev] Azure event hub network access I am getting http error 404. I am able to access the site via telnet which eliminates network issues. Here is the code and subsequent errors user/bin/python import sys import azure import socket from azure.servicebus import ( _service_bus_error_handler ) from azure.servicebus.servicebusservice import ( ServiceBusService, ServiceBusSASAuthentication ) from azure.http import ( HTTPRequest, HTTPError ) from azure.http.httpclient import _HTTPClient class EventHubClient(object): def sendMessage(self,body,partition): eventHubHost = "pac-ns.servicebus.windows.net" httpclient = _HTTPClient(service_instance=self) sasKeyName = "pac-pl" sasKeyValue = "IhkEepQPLfSy9jo6H2Yxxxxxxxxxxxxxxxxxxxx=" authentication = ServiceBusSASAuthentication(sasKeyName,sasKeyValue) request = HTTPRequest() request.method = "POST" request.host = eventHubHost request.protocol_override = "https" # request.path = "/myhub/publishers/" + partition + "/messages?api-version=20 14-05" request.body = body request.headers.append(('Content-Type', 'application/atom+xml;type=entry;cha rset=utf-8')) authentication.sign_request(request, httpclient) request.headers.append(('Content-Length', str(len(request.body)))) status = 0 try: resp = httpclient.perform_request(request) status = resp.status except HTTPError as ex: status = ex.status return status class EventDataParser(object): def getMessage(self,payload,sensorId): host = socket.gethostname() body = "{ \"DeviceId\" : \"" + host + "\",\"SensorData\": [ " msgs = payload.split(",") first = True for msg in msgs: # print msg sensorType = msg.split(":")[0] sensorValue = msg.split(":")[1] if first == True: first = False else: body += "," body += "{ \"SensorId\" : \"" + sensorId + "\", \"SensorType\" : \"" + sen sorType + "\", \"SensorValue\" : " + sensorValue + " }" body += "]}" return body hubClient = EventHubClient() parser = EventDataParser() hostname = socket.gethostname() sensor = sys.argv[2] body = parser.getMessage(sys.argv[1],sensor) hubStatus = hubClient.sendMessage(body,hostname) # return the HTTP status to the caller print hubStatus print hostname print sensor ~/IOT/AZURE$ python send.py temperature:22,humidity:20 deviceid 404 ubuntu deviceid { "DeviceId" : "ubuntu","SensorData": [ { "SensorId" : "deviceid", "SensorType" : "temperature", "SensorValue" : 22 },{ "SensorId" : "deviceid", "SensorType" : "humidity", "SensorValue" : 20 }]} -- Syed Khalid -------------- next part -------------- An HTML attachment was scrubbed... URL: From ole-usenet-spam at gmx.net Fri Feb 6 03:27:05 2015 From: ole-usenet-spam at gmx.net (=?utf-8?B?0J5s0LUg0IV0ctC10ZbRgWjQtXI=?=) Date: Fri, 06 Feb 2015 09:27:05 +0100 Subject: multiprocessing.Queue() and missing sem_open References: <877fvwaz1y.fsf@news.ole.ath.cx> Message-ID: Chris Angelico writes: > On Fri, Feb 6, 2015 at 7:22 AM, ?l? ?tr???h?r wrote: >> I am just trying to prepare a package (astropy) for (Debian) Hurd. This >> os lacks a sem_open() implementation. When I now try: >> >> import multiprocessing >> q = multiprocessing.Queue() >> >> I get an ImportError with Python 2.7, but an AttributeError with Python >> 3.4. In the documentation of multiprocessing.Queue() I couldn't find any >> hint that it would throw this exception. > > Neither of those errors is being consciously thrown by the Queue > class; the latter means that multiprocessing exists but it has no > Queue in it, and the former means that the entire multiprocessing > module is absent. Trace with 3.4.2: self = , maxsize = 0 def Queue(self, maxsize=0): '''Returns a queue object''' from .queues import Queue > return Queue(maxsize, ctx=self.get_context()) /usr/lib/python3.4/multiprocessing/context.py:101: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , maxsize = 0 def __init__(self, maxsize=0, *, ctx): if maxsize <= 0: > maxsize = _multiprocessing.SemLock.SEM_VALUE_MAX E AttributeError: 'module' object has no attribute 'SemLock' /usr/lib/python3.4/multiprocessing/queues.py:38: AttributeError Trace with 2.7.9: maxsize = 0 def Queue(maxsize=0): ''' Returns a queue object ''' > from multiprocessing.queues import Queue /usr/lib/python2.7/multiprocessing/__init__.py:217: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __all__ = ['Queue', 'SimpleQueue', 'JoinableQueue'] import sys import os import threading import collections import time import atexit import weakref from Queue import Empty, Full import _multiprocessing from multiprocessing import Pipe > from multiprocessing.synchronize import Lock, BoundedSemaphore, Semaphore, Condition /usr/lib/python2.7/multiprocessing/queues.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Condition', 'Event' ] import threading import os import sys from time import time as _time, sleep as _sleep import _multiprocessing from multiprocessing.process import current_process from multiprocessing.util import Finalize, register_after_fork, debug from multiprocessing.forking import assert_spawning, Popen # Try to import the mp.synchronize module cleanly, if it fails # raise ImportError for platforms lacking a working sem_open implementation. # See issue 3770 try: from _multiprocessing import SemLock except (ImportError): raise ImportError("This platform lacks a functioning sem_open" + " implementation, therefore, the required" + " synchronization primitives needed will not" + > " function, see issue 3770.") E ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. /usr/lib/python2.7/multiprocessing/synchronize.py:59: ImportError Both are with the same Hurd version. > I'm guessing you built Python from source? No, I am using the precompiled version that comes with Debian. It has no specific patch for sem_open. > But this is the exact way that Python will normally signal that > something isn't available. If your platform doesn't have, say, > os.O_BINARY, because that flag has no meaning for you, then you don't > get NotImplementedError - you simply get AttributeError. As you can see from the trace, multiprocessing.Queue is there, but initialization fails. >> Can I be sure that the following works also in future? >> >> try >> q = multiprocessing.Queue() >> except (ImportError, AttributeError) >> # handle the case of missing sem_open >> >> Or what is the correct way to catch a not working Queue caused by a >> missing sem_open() implementation? > > The ImportError will come from the import statement No. > the AttributeError will come from the attribute lookup - the above code > can't[1] bomb out with ImportError. Maybe it can't but it actually does. > So a more correct way would be something like: > > try: > import multiprocessing > multiprocessing.Queue > except (ImportError, AttributeError): > # handle the absence Is it sure that this will work in the future as well? > Or alternatively, don't even bother to try/except the import. If your > code requires multiprocessing.Queue (as opposed to just queue.Queue), > chances are you can't cope with the absence of the module. This code is used for a fast, asynchronious reader. If it is not implemented, there is a (slower) fallback solution. However, we just want to catch the "is not implemented" case here but not any other problem that may appear during opening or reading the file. Shall I file a bug for this? Best Ole From rosuav at gmail.com Fri Feb 6 03:34:08 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Feb 2015 19:34:08 +1100 Subject: multiprocessing.Queue() and missing sem_open In-Reply-To: References: <877fvwaz1y.fsf@news.ole.ath.cx> Message-ID: On Fri, Feb 6, 2015 at 7:27 PM, ?l? ?tr???h?r wrote: >> the AttributeError will come from the attribute lookup - the above code >> can't[1] bomb out with ImportError. > > Maybe it can't but it actually does. Huh. Okay, my bad. (I don't have a Hurd system to test on, all my Debians are Linux.) Sorry for the misinformation. >> So a more correct way would be something like: >> >> try: >> import multiprocessing >> multiprocessing.Queue >> except (ImportError, AttributeError): >> # handle the absence > > Is it sure that this will work in the future as well? Frankly, I don't know now. It's reasonably likely that it will, but in the absence of actual documentation, I couldn't say. ChrisA From fabiofz at gmail.com Fri Feb 6 05:35:04 2015 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Fri, 6 Feb 2015 08:35:04 -0200 Subject: PyDev 3.9.2 Released Message-ID: What is PyDev? --------------------------- PyDev is an open-source Python IDE on top of Eclipse for Python, Jython and IronPython development. It comes with goodies such as code completion, syntax highlighting, syntax analysis, code analysis, refactor, debug, interactive console, etc. Details on PyDev: http://pydev.org Details on its development: http://pydev.blogspot.com What is LiClipse? --------------------------- LiClipse is a PyDev standalone with goodies such as support for Multiple cursors, theming and a number of other languages such as Django Templates, Kivy Language, Mako Templates, Html, Javascript, etc. It's also a commercial counterpart which helps supporting the development of PyDev. Details on LiClipse: http://www.liclipse.com/ Release Highlights: ------------------------------- * **Important**: PyDev requires Eclipse 3.8 or 4.3 onwards and Java 7! For older versions, keep using PyDev 2.x (use LiClipse: http://www.liclipse.com for a PyDev standalone with all requirements bundled). * **PyVmMonitor Integration** * PyVmMonitor: http://www.pyvmmonitor.com/ is now in public beta, so, the PyDev integration (Window > Show View > Other > PyDev > Profile) may be used to profile your programs. * **Debugger** * The debug view now has an interactive console (with history) attached to it by default (which may be toggled on/off). (PyDev-507) * Debugger no longer reopens a file when that file is already opened. (PyDev-456) * Handled issue when getting referrers for some object gave an error if it was found in a dict where the key is not a string. * When interactive console starts in debug session, a banner is no longer shown. * Stepping with #@DontTrace no longer returns through decorator call-site. (PyDev-526) * The default for tracing template render exceptions on Django is now false. * **Interactive Console** * F2 to send contents from editor to console now considers backslash continuations. (PyDev-502) * Interactive Console interrupt now properly interrupts a sleep call (when possible). (PyDev-500) * PyDev interactive console now has a user-specified encoding (by default UTF-8). (PyDev-454) * Scroll the console on stdout / stderr output. (PyDev-504, patch by James Blackburn) * Moved interactive console initial commands to a separate preferences page. * Handling interrupted system call EINTR in the pydevconsole.py. (PyDev-534) * Fixed racing condition where the output of the console could appear as a user input. (PyDev-490, patch by James Blackburn) * **Refactoring** * Fixed issue where indentation lost on rename module refactoring. (PyDev-498) * The rename modules refactoring wizard now provides a way to do a simple resource rename (to rename extensions). * **Others** * Converting filename from .pyx to .py doesn't loose indexing on the file anymore. (PyDev-525) * The Cython parser now properly scopes methods. * Pasting contents directly in the PyDev package explorer to create a file uses the proper delimiter. * Fixed deadlock in ImageCache when rendering debug completions from console. (PyDev-527) * Fixed deadlock on racing condition when rendering PyTextHover. (PyDev-523) * Tab settings were separated from the editor color settings and may now be persisted in the project/user settings. * Fixed surround with try..finally/except indentation on Ctrl+1 when some line has a comment which has a different indentation. Cheers, -- Fabio Zadrozny ------------------------------------------------------ Software Developer LiClipse http://www.liclipse.com PyDev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com PyVmMonitor - Python Profiler http://www.pyvmmonitor.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jennywerba78 at gmail.com Fri Feb 6 06:29:18 2015 From: jennywerba78 at gmail.com (Jocel23) Date: Fri, 6 Feb 2015 03:29:18 -0800 (PST) Subject: Booksigning Party at PyCon This Year! In-Reply-To: <43E5D0F3.5060305@taupro.com> References: <43E5D0F3.5060305@taupro.com> Message-ID: <1423222158070-5085422.post@n6.nabble.com> Well dear, in my opinion attire should be according to the occasion and wedding reception is a special day for couple. And every couple wants that their guests love the event. Rest is up to you my friend. -- View this message in context: http://python.6.x6.nabble.com/Booksigning-Party-at-PyCon-This-Year-tp935988p5085422.html Sent from the Python - python-list mailing list archive at Nabble.com. From jeanpierreda at gmail.com Fri Feb 6 08:09:05 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Fri, 6 Feb 2015 05:09:05 -0800 Subject: meaning of: line, = In-Reply-To: <98777b0d-8f6c-4e2d-965f-0ef875e962b0@googlegroups.com> References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> <98777b0d-8f6c-4e2d-965f-0ef875e962b0@googlegroups.com> Message-ID: Sorry for late reply, I somehow missed this email. On Thu, Feb 5, 2015 at 8:59 AM, Rustom Mody wrote: > The reason I ask: I sorely miss haskell's pattern matching in python. > > It goes some way: > >>>> ((x,y),z) = ((1,2),3) >>>> x,y,z > (1, 2, 3) > > But not as far as I would like: > >>>> ((x,y),3) = ((1,2),3) > File "", line 1 > SyntaxError: can't assign to literal >>>> > > [Haskell] > > Prelude> let (x, (y, (42, z, "Hello"))) = (1, (2, (42, 3, "Hello"))) > Prelude> (x,y,z) > (1,2,3) Yeah, but Haskell is ludicrous. Prelude> let (x, 2) = (1, 3) Prelude> Only non-falsifiable patterns really make sense as the left hand side of an assignment in a language without exceptions, IMO. Otherwise you should use a match/case statement. (Of course, Python does have exceptions...) -- Devin From rustompmody at gmail.com Fri Feb 6 08:20:51 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 6 Feb 2015 05:20:51 -0800 (PST) Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> <98777b0d-8f6c-4e2d-965f-0ef875e962b0@googlegroups.com> Message-ID: <31a513ab-3eac-4664-9a11-0b0028197c42@googlegroups.com> On Friday, February 6, 2015 at 6:40:23 PM UTC+5:30, Devin Jeanpierre wrote: > Sorry for late reply, I somehow missed this email. > > On Thu, Feb 5, 2015 at 8:59 AM, Rustom Mody wrote: > > The reason I ask: I sorely miss haskell's pattern matching in python. > > > > It goes some way: > > > >>>> ((x,y),z) = ((1,2),3) > >>>> x,y,z > > (1, 2, 3) > > > > But not as far as I would like: > > > >>>> ((x,y),3) = ((1,2),3) > > File "", line 1 > > SyntaxError: can't assign to literal > >>>> > > > > [Haskell] > > > > Prelude> let (x, (y, (42, z, "Hello"))) = (1, (2, (42, 3, "Hello"))) > > Prelude> (x,y,z) > > (1,2,3) > > Yeah, but Haskell is ludicrous. > > Prelude> let (x, 2) = (1, 3) > Prelude> > > Only non-falsifiable patterns really make sense as the left hand side > of an assignment in a language without exceptions, IMO. Otherwise you > should use a match/case statement. Which is what people do 99% of the times - write pattern matching function defs I was just writing a 1 liner showing matching of constants and variables simultaneously From johnnybureddu at gmx.com Fri Feb 6 08:55:57 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Fri, 6 Feb 2015 05:55:57 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From rustompmody at gmail.com Fri Feb 6 09:03:53 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 6 Feb 2015 06:03:53 -0800 (PST) Subject: meaning of: line, = In-Reply-To: References: <07f4498c-7917-4315-85d4-73283a830f2d@googlegroups.com> <54d33aa6$0$13003$c3e8da3$5496439d@news.astraweb.com> <6304fe65-0606-40eb-8732-c9ba9ae168e7@googlegroups.com> <98777b0d-8f6c-4e2d-965f-0ef875e962b0@googlegroups.com> Message-ID: <4ee49856-d2f8-471f-862d-1720a452ad4f@googlegroups.com> On Friday, February 6, 2015 at 6:40:23 PM UTC+5:30, Devin Jeanpierre wrote: > Sorry for late reply, I somehow missed this email. > > On Thu, Feb 5, 2015 at 8:59 AM, Rustom Mody wrote: > > The reason I ask: I sorely miss haskell's pattern matching in python. > > > > It goes some way: > > > >>>> ((x,y),z) = ((1,2),3) > >>>> x,y,z > > (1, 2, 3) > > > > But not as far as I would like: > > > >>>> ((x,y),3) = ((1,2),3) > > File "", line 1 > > SyntaxError: can't assign to literal > >>>> > > > > [Haskell] > > > > Prelude> let (x, (y, (42, z, "Hello"))) = (1, (2, (42, 3, "Hello"))) > > Prelude> (x,y,z) > > (1,2,3) > > Yeah, but Haskell is ludicrous. > > Prelude> let (x, 2) = (1, 3) > Prelude> > > Only non-falsifiable patterns really make sense as the left hand side > of an assignment in a language without exceptions, IMO. Otherwise you > should use a match/case statement. (Of course, Python does have > exceptions...) Also its good to see the full context of your example: Prelude> let (x, 2) = (1, 3) Prelude> x *** Exception: :2:5-19: Irrefutable pattern failed for pattern (x, 2) Prelude> So I am not sure what you find ludicrous. Haskell is a lazy language. This is lazy behavior. What else can it be? [Does not make me a great fan of laziness -- which is another matter] From jessemuzziddu at mail.com Fri Feb 6 09:34:22 2015 From: jessemuzziddu at mail.com (CHE SIA REVOLUCIOOON!!!) Date: Fri, 6 Feb 2015 06:34:22 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <018343a6-fb14-4c8d-ade3-42985671470a@googlegroups.com> -- redacted -- From jessemuzziddu at mail.com Fri Feb 6 09:35:19 2015 From: jessemuzziddu at mail.com (CHE SIA REVOLUCIOOON!!!) Date: Fri, 6 Feb 2015 06:35:19 -0800 (PST) Subject: -- redacted -- In-Reply-To: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> References: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> Message-ID: <7619bc39-8241-43be-9416-8196ba3def31@googlegroups.com> -- redacted -- From jessemuzziddu at mail.com Fri Feb 6 09:41:55 2015 From: jessemuzziddu at mail.com (CHE SIA REVOLUCIOOON!!!) Date: Fri, 6 Feb 2015 06:41:55 -0800 (PST) Subject: -- redacted -- In-Reply-To: <04938df1-8d59-4262-acff-648c1475d0f5@googlegroups.com> References: <04938df1-8d59-4262-acff-648c1475d0f5@googlegroups.com> Message-ID: -- redacted -- From ndbecker2 at gmail.com Fri Feb 6 09:55:53 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 06 Feb 2015 09:55:53 -0500 Subject: help with pypeg2? Message-ID: Trying out pypeg2. The below grammar is recursive. A 'Gen' is an ident followed by parenthesized args. args is a csl of alphanum or Gen. The tests 'p' and 'p2' are fine, but 'p3' fails SyntaxError: expecting u')' from __future__ import unicode_literals, print_function from pypeg2 import * ident = re.compile (r'[a-z]+') alphanum = re.compile (r'[a-z0-9]+') num = re.compile (r'[0-9]+') class args (List): grammar = maybe_some ( csl ([alphanum, Gen])) class Gen (List): grammar = attr ('type', ident), '(', attr ('args', args), ')' p = parse ('abc,123', args) p2 = parse ('abc(123,456)', Gen) p3 = parse ('abc(123,def(456))', Gen) -- -- Those who don't understand recursion are doomed to repeat it From invalid at invalid.invalid Fri Feb 6 10:38:21 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 6 Feb 2015 15:38:21 +0000 (UTC) Subject: Usage of some pastebin service proposed References: Message-ID: On 2015-02-05, Abhiram R wrote: > I've noticed a lot of people enquiring about syntactic errors and email > somewhat butchers the indentation every now and then Just use a non-broken email program. -- Grant Edwards grant.b.edwards Yow! Are we laid back yet? at gmail.com From illusiontechniques at gmail.com Fri Feb 6 11:34:00 2015 From: illusiontechniques at gmail.com (C Smith) Date: Fri, 6 Feb 2015 08:34:00 -0800 Subject: Matplotlib import error Message-ID: I had python 2.7.6 installed on OS X yosemite, which has always worked fine, until I tried to install matplotlib with pip. I got the same error below and upgraded to 2.7.9, used pip to upgrade all the packages, but still get the same error. >>> import matplotlib Traceback (most recent call last): File "", line 1, in File "/Library/Python/2.7/site-packages/matplotlib/__init__.py", line 180, in from matplotlib.cbook import is_string_like File "/Library/Python/2.7/site-packages/matplotlib/cbook.py", line 33, in import numpy as np File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.py", line 170, in from . import add_newdocs File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in from numpy.lib import add_newdoc File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/__init__.py", line 18, in from .polynomial import * File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 19, in from numpy.linalg import eigvals, lstsq, inv File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 51, in from .linalg import * File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 29, in from numpy.linalg import lapack_lite, _umath_linalg ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so, 2): Symbol not found: __gfortran_compare_string Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so Expected in: flat namespace in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so From ian.g.kelly at gmail.com Fri Feb 6 11:40:10 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 6 Feb 2015 09:40:10 -0700 Subject: help with pypeg2? In-Reply-To: References: Message-ID: On Fri, Feb 6, 2015 at 7:55 AM, Neal Becker wrote: > Trying out pypeg2. The below grammar is recursive. A 'Gen' is an ident > followed by parenthesized args. args is a csl of alphanum or Gen. > > The tests 'p' and 'p2' are fine, but 'p3' fails > SyntaxError: expecting u')' > > > from __future__ import unicode_literals, print_function > from pypeg2 import * > > ident = re.compile (r'[a-z]+') > alphanum = re.compile (r'[a-z0-9]+') > num = re.compile (r'[0-9]+') > > class args (List): > grammar = maybe_some ( csl ([alphanum, Gen])) I'm not familiar with pypeg2, but should this use optional instead of maybe_some? The csl function already produces a list, so the result of maybe_some on that would be one or more consecutive lists. Also, it looks from the docs like it should just be "csi(alphanum, Gen)" (no list in the arguments). From rosuav at gmail.com Fri Feb 6 13:20:32 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Feb 2015 05:20:32 +1100 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On Sat, Feb 7, 2015 at 3:34 AM, C Smith wrote: > ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so, > 2): Symbol not found: __gfortran_compare_string Ah, installing from source on a Mac and having problems. Have I heard this story before? Just occasionally? It's almost as if the platform is actively hostile toward C and Fortran... But I found this at the far end of a Google search. Maybe it'll help? http://stackoverflow.com/questions/22080154/error-importing-numpy-after-upgrading-with-pip ChrisA From ndbecker2 at gmail.com Fri Feb 6 13:26:44 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 06 Feb 2015 13:26:44 -0500 Subject: help with pypeg2? References: Message-ID: Ian Kelly wrote: > On Fri, Feb 6, 2015 at 7:55 AM, Neal Becker wrote: >> Trying out pypeg2. The below grammar is recursive. A 'Gen' is an ident >> followed by parenthesized args. args is a csl of alphanum or Gen. >> >> The tests 'p' and 'p2' are fine, but 'p3' fails >> SyntaxError: expecting u')' >> >> >> from __future__ import unicode_literals, print_function >> from pypeg2 import * >> >> ident = re.compile (r'[a-z]+') >> alphanum = re.compile (r'[a-z0-9]+') >> num = re.compile (r'[0-9]+') >> >> class args (List): >> grammar = maybe_some ( csl ([alphanum, Gen])) > > I'm not familiar with pypeg2, but should this use optional instead of > maybe_some? The csl function already produces a list, so the result of > maybe_some on that would be one or more consecutive lists. > > Also, it looks from the docs like it should just be "csi(alphanum, > Gen)" (no list in the arguments). It didn't work without [list..] (even trivial examples did not work without it). I don't know if it's broken, I though csl (*args) should mean a comma-sep-list of _any_ of the *args, but it doesn't work. [list] means alternatives, and does seem to work (for simple examples). -- -- Those who don't understand recursion are doomed to repeat it From illusiontechniques at gmail.com Fri Feb 6 13:49:39 2015 From: illusiontechniques at gmail.com (C Smith) Date: Fri, 6 Feb 2015 10:49:39 -0800 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On Fri, Feb 6, 2015 at 10:20 AM, Chris Angelico wrote: > On Sat, Feb 7, 2015 at 3:34 AM, C Smith wrote: >> ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so, >> 2): Symbol not found: __gfortran_compare_string > > Ah, installing from source on a Mac and having problems. Have I heard > this story before? Just occasionally? It's almost as if the platform > is actively hostile toward C and Fortran... > > But I found this at the far end of a Google search. Maybe it'll help? > > http://stackoverflow.com/questions/22080154/error-importing-numpy-after-upgrading-with-pip > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list I already had gcc installed and had used it to build other packages. It worked in the past when clang was throwing errors. I tried making the symlink from the stackoverflow question, but still get the same errors. I will look around and see if I can find if gcc is even being used by pip properly. I really regret not using virtual environments from day one. Lesson learned the hard way, I guess. Thanks From ian.g.kelly at gmail.com Fri Feb 6 14:02:52 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 6 Feb 2015 12:02:52 -0700 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On Fri, Feb 6, 2015 at 11:49 AM, C Smith wrote: > On Fri, Feb 6, 2015 at 10:20 AM, Chris Angelico wrote: >> On Sat, Feb 7, 2015 at 3:34 AM, C Smith wrote: >>> ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so, >>> 2): Symbol not found: __gfortran_compare_string >> >> Ah, installing from source on a Mac and having problems. Have I heard >> this story before? Just occasionally? It's almost as if the platform >> is actively hostile toward C and Fortran... >> >> But I found this at the far end of a Google search. Maybe it'll help? >> >> http://stackoverflow.com/questions/22080154/error-importing-numpy-after-upgrading-with-pip >> >> ChrisA >> -- >> https://mail.python.org/mailman/listinfo/python-list > > I already had gcc installed and had used it to build other packages. > It worked in the past when clang was throwing errors. I tried making > the symlink from the stackoverflow question, but still get the same > errors. I will look around and see if I can find if gcc is even being > used by pip properly. I really regret not using virtual environments > from day one. Lesson learned the hard way, I guess. Thanks With numpy and non-Linux platforms, I find it saves a lot of pain to just install a binary distribution. From breamoreboy at yahoo.co.uk Fri Feb 6 14:12:56 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 06 Feb 2015 19:12:56 +0000 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On 06/02/2015 19:02, Ian Kelly wrote: > On Fri, Feb 6, 2015 at 11:49 AM, C Smith wrote: >> On Fri, Feb 6, 2015 at 10:20 AM, Chris Angelico wrote: >>> On Sat, Feb 7, 2015 at 3:34 AM, C Smith wrote: >>>> ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so, >>>> 2): Symbol not found: __gfortran_compare_string >>> >>> Ah, installing from source on a Mac and having problems. Have I heard >>> this story before? Just occasionally? It's almost as if the platform >>> is actively hostile toward C and Fortran... >>> >>> But I found this at the far end of a Google search. Maybe it'll help? >>> >>> http://stackoverflow.com/questions/22080154/error-importing-numpy-after-upgrading-with-pip >>> >>> ChrisA >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >> >> I already had gcc installed and had used it to build other packages. >> It worked in the past when clang was throwing errors. I tried making >> the symlink from the stackoverflow question, but still get the same >> errors. I will look around and see if I can find if gcc is even being >> used by pip properly. I really regret not using virtual environments >> from day one. Lesson learned the hard way, I guess. Thanks > > With numpy and non-Linux platforms, I find it saves a lot of pain to > just install a binary distribution. > Regardless of numpy this site http://www.lfd.uci.edu/~gohlke/pythonlibs/ is a must know about for Windows users. Is there anything like it for the Mac? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rgaddi at technologyhighland.invalid Fri Feb 6 16:17:34 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Fri, 6 Feb 2015 21:17:34 +0000 (UTC) Subject: Setuptools, __init__ and __main__ Message-ID: So I'm trying to wrap my head around packaging issues, and according to some quick Google searches I'm not the only one. I'm developing applications of various size for internal deployment. When I'm actually getting the whole thing written, I've got my initialization stub in __main__.py, because it's an application rather than a library. But then when I migrated over to setuptools I had to switch it to __init__.py and change around all my import statements to compensate. Then I can reference it as an entry point. This feels silly. I'm sure I could go stumbling around until I come up with an elegant solution, but does anyone already have one? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ben+python at benfinney.id.au Fri Feb 6 16:35:59 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 07 Feb 2015 08:35:59 +1100 Subject: Setuptools, __init__ and __main__ References: Message-ID: <85lhkag1ts.fsf@benfinney.id.au> Rob Gaddi writes: > So I'm trying to wrap my head around packaging issues Congratulations on tackling this. You will likely find the Python Packaging User Guide helpful. Also know that Python packaging was in a terrible state for a long stretch of years, but now the Python Packaging Authority exists which has tended to steadily improve the ecosystem in recent years. > I'm developing applications of various size for internal deployment. > When I'm actually getting the whole thing written, I've got my > initialization stub in __main__.py, because it's an application rather > than a library. But then when I migrated over to setuptools I had to > switch it to __init__.py and change around all my import statements to > compensate. Then I can reference it as an entry point. Yes, it's one of the enduring bad parts of Python that it makes the command-line invocation of an application overly awkward. What do you mean by ?initialization stub?? That could mean a lot of different things, and I don't want to guess what you mean. As for the difference between library versus application: You should be writing all your code as discretely-testable units anyway, and unit tests should be importing those in order to test them. So the whole lot needs to be in a package hierarchy of some form in order to do that. That said, I haven't made use of the ?execute a directory as a program? Python feature, so I might be misunderstanding what the problem is. > This feels silly. I'm sure I could go stumbling around until I come up > with an elegant solution, but does anyone already have one? It seems to me the problem is avoidable by ensuring from the start that all your code is importable by your unit tests; that way, it's ready for a Setuptools entry point declaration without changes. -- \ ?Quidquid latine dictum sit, altum viditur.? (?Whatever is | `\ said in Latin, sounds profound.?) ?anonymous | _o__) | Ben Finney From davea at davea.name Fri Feb 6 16:44:26 2015 From: davea at davea.name (Dave Angel) Date: Fri, 06 Feb 2015 16:44:26 -0500 Subject: Setuptools, __init__ and __main__ In-Reply-To: <85lhkag1ts.fsf@benfinney.id.au> References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: <54D535BA.3000809@davea.name> On 02/06/2015 04:35 PM, Ben Finney wrote: > Rob Gaddi writes: > >> So I'm trying to wrap my head around packaging issues > > Congratulations on tackling this. You will likely find the Python > Packaging User Guide helpful. > > Also know that Python packaging was in a terrible state for a long > stretch of years, but now the Python Packaging Authority exists > which has tended to steadily improve the > ecosystem in recent years. > >> I'm developing applications of various size for internal deployment. >> When I'm actually getting the whole thing written, I've got my >> initialization stub in __main__.py, because it's an application rather >> than a library. But then when I migrated over to setuptools I had to >> switch it to __init__.py and change around all my import statements to >> compensate. Then I can reference it as an entry point. > > Yes, it's one of the enduring bad parts of Python that it makes the > command-line invocation of an application overly awkward. > > What do you mean by ?initialization stub?? That could mean a lot of > different things, and I don't want to guess what you mean. > > As for the difference between library versus application: You should be > writing all your code as discretely-testable units anyway, and unit > tests should be importing those in order to test them. So the whole lot > needs to be in a package hierarchy of some form in order to do that. > > That said, I haven't made use of the ?execute a directory as a program? > Python feature, so I might be misunderstanding what the problem is. > >> This feels silly. I'm sure I could go stumbling around until I come up >> with an elegant solution, but does anyone already have one? > > It seems to me the problem is avoidable by ensuring from the start that > all your code is importable by your unit tests; that way, it's ready for > a Setuptools entry point declaration without changes. > To this I'd like to add the caveat that both __init__.py and __main__.py are reserved names, and putting arbitrary code in either one is quite risky. I suspect you're putting library code in your application code file, and naming it __main__ because that way you can cheat around the recursive import restrictions. Please don't do that. Organize your code so that there are no loops in the import library hierarchy. If that means your application file is only six lines, so be it. And don't name any source code __main__.py, Finally, the mere presence of __init__.py has a special meaning, and code in it should probably be limited to doing namespace munging that a package may require to keep its public interface simpler. -- DaveA From rgaddi at technologyhighland.invalid Fri Feb 6 17:20:10 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Fri, 6 Feb 2015 22:20:10 +0000 (UTC) Subject: Setuptools, __init__ and __main__ References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: On Fri, 06 Feb 2015 16:44:26 -0500, Dave Angel wrote: > On 02/06/2015 04:35 PM, Ben Finney wrote: >> Rob Gaddi writes: >> >>> So I'm trying to wrap my head around packaging issues >> >> Congratulations on tackling this. You will likely find the Python >> Packaging User Guide helpful. >> >> Also know that Python packaging was in a terrible state for a long >> stretch of years, but now the Python Packaging Authority exists >> which has tended to steadily improve the >> ecosystem in recent years. >> >>> I'm developing applications of various size for internal deployment. >>> When I'm actually getting the whole thing written, I've got my >>> initialization stub in __main__.py, because it's an application rather >>> than a library. But then when I migrated over to setuptools I had to >>> switch it to __init__.py and change around all my import statements to >>> compensate. Then I can reference it as an entry point. >> >> Yes, it's one of the enduring bad parts of Python that it makes the >> command-line invocation of an application overly awkward. >> >> What do you mean by ?initialization stub?? That could mean a lot of >> different things, and I don't want to guess what you mean. >> >> As for the difference between library versus application: You should be >> writing all your code as discretely-testable units anyway, and unit >> tests should be importing those in order to test them. So the whole lot >> needs to be in a package hierarchy of some form in order to do that. >> >> That said, I haven't made use of the ?execute a directory as a program? >> Python feature, so I might be misunderstanding what the problem is. >> >>> This feels silly. I'm sure I could go stumbling around until I come up >>> with an elegant solution, but does anyone already have one? >> >> It seems to me the problem is avoidable by ensuring from the start that >> all your code is importable by your unit tests; that way, it's ready >> for a Setuptools entry point declaration without changes. >> >> > To this I'd like to add the caveat that both __init__.py and __main__.py > are reserved names, and putting arbitrary code in either one is quite > risky. > Not so much risky as just, there's a thing that it does. > I suspect you're putting library code in your application code file, and > naming it __main__ because that way you can cheat around the recursive > import restrictions. Please don't do that. > > Organize your code so that there are no loops in the import library > hierarchy. If that means your application file is only six lines, so be > it. > > And don't name any source code __main__.py, > > Finally, the mere presence of __init__.py has a special meaning, and > code in it should probably be limited to doing namespace munging that a > package may require to keep its public interface simpler. So, I'll take the app I'm working on right now as an example. I've got 348 lines in my __init__.py, and another 200 in another library. Little baby program. My library code isn't in __init__.py (or previously in __main__) because I'm trying to dodge recursive imports; it's there because the code is short enough and tightly-coupled enough that I didn't see upside in splitting any of it out to a separate file. Under other circumstances when the app has become substantial, the bulk of the application code winds up in, e.g. mainwindow.py, and the __main__ does nothing but call mainwindow.main(). The advice to reduce the entry point code (be it __init__ or __main__) to a small stub and move the bulk of the code to another library doesn't change the underlying situation. Setuptools seems to require that the application be a "package", i.e. a thing that has an __init__.py. But the python application start mechanism seems to be steering me to using a __main__. I found a recommendation at https://chriswarrick.com/blog/2014/09/15/ python-apps-the-right-way-entry_points-and-scripts/ that seems to say that I should get around this by simply having an empty __init__. I guess I can move the program to foobar.py, have an empty __init__, a stub __main__ that just calls foobar.main(), and an entry-point of foobar.foobar:main . It just feels like a bit of a heavy solution to have two unnecessary files. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ethan at stoneleaf.us Fri Feb 6 17:51:29 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 06 Feb 2015 14:51:29 -0800 Subject: Setuptools, __init__ and __main__ In-Reply-To: References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: <54D54571.3060706@stoneleaf.us> On 02/06/2015 02:20 PM, Rob Gaddi wrote: > > My library code isn't in __init__.py (or previously in __main__) because > I'm trying to dodge recursive imports; it's there because the code is > short enough and tightly-coupled enough that I didn't see upside in > splitting any of it out to a separate file. Under other circumstances > when the app has become substantial, the bulk of the application code > winds up in, e.g. mainwindow.py, and the __main__ does nothing but call > mainwindow.main(). Sounds like you would have been okay to keep your code in __main__.py and have an empty __init__.py. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From p.f.moore at gmail.com Fri Feb 6 17:54:07 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Feb 2015 14:54:07 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> Message-ID: <51d12f89-7d03-41e9-a2c9-3960a15ed6aa@googlegroups.com> On Thursday, 5 February 2015 23:02:42 UTC, Paul Moore wrote: > Nice! Thanks both of you. I start to see how to think about these > things now. I'll play around with some of the other examples I've got, > and see how far I get. Just a quick status update, in case you're interested. With relatively little work (considering I started not knowing much about numpy) I managed to put together solutions for a couple of my friend's problems which ran in basically the same time as his custom C code. Very impressive! Thanks again for the help. Paul From ben+python at benfinney.id.au Fri Feb 6 17:56:20 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 07 Feb 2015 09:56:20 +1100 Subject: Setuptools, __init__ and __main__ References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: <85h9uyfy3v.fsf@benfinney.id.au> Rob Gaddi writes: > So, I'll take the app I'm working on right now as an example. I've got > 348 lines in my __init__.py, and another 200 in another library. > Little baby program. By ?another library?, I think you mean ?another module?. I'll assume that in the rest of this response. > My library code isn't in __init__.py (or previously in __main__) > because I'm trying to dodge recursive imports; it's there because the > code is short enough and tightly-coupled enough that I didn't see > upside in splitting any of it out to a separate file. Sure, there's no need to make more modules if the code is small and coherent. But you describe it as ?tightly-coupled?; that's a warning sign already. How do you automate tests of its individual units? The code should be in separate functions with tightly-defined scope and narrow, well-specified interfaces. > Under other circumstances when the app has become substantial, the > bulk of the application code winds up in, e.g. mainwindow.py, and the > __main__ does nothing but call mainwindow.main(). Great! So you already have an entry point for Setuptools: the entry point is something like "fooapp.mainwindow:main". > The advice to reduce the entry point code (be it __init__ or __main__) > to a small stub and move the bulk of the code to another [module] > doesn't change the underlying situation. Setuptools seems to require > that the application be a "package", i.e. a thing that has an > __init__.py. But the python application start mechanism seems to be > steering me to using a __main__. It's not Setuptools per se, it's Python's import mechanism. It is a deliberate design decision that direct import of a module makes that module blind to its location in the package hierarchy. That's a design decision I deplore, because it makes something that should be easy (write a command directly into a file and invoke that file as an executable program) tortuously difficult when the program comprises several modules. > I found a recommendation at https://chriswarrick.com/blog/2014/09/15/ > python-apps-the-right-way-entry_points-and-scripts/ that seems to say > that I should get around this by simply having an empty __init__. More importantly, the core tip there is that you can write your entry point as "fooapp.__main__:main". > I guess I can move the program to foobar.py, have an empty __init__, a > stub __main__ that just calls foobar.main(), and an entry-point of > foobar.foobar:main . It just feels like a bit of a heavy solution to > have two unnecessary files. Well, they're small but not unnecessary. They're explicitly telling Python the meaning of a particular directory, rather than leaving it to guess. The quirks of Python's import system, and especially the implications for trying to just get ?./fooapp? working, are covered in a ?here's some traps to avoid? article by Nick Coghlan . I really wish Python had a better solution for this, but it doesn't. This is one of the few downsides of Python's design IMO, and we just need to work with it. -- \ ?I find the whole business of religion profoundly interesting. | `\ But it does mystify me that otherwise intelligent people take | _o__) it seriously.? ?Douglas Adams | Ben Finney From p.f.moore at gmail.com Fri Feb 6 18:03:56 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Feb 2015 15:03:56 -0800 (PST) Subject: Setuptools, __init__ and __main__ In-Reply-To: References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: <5e9d887f-826e-485a-a61e-a511ff6b50f8@googlegroups.com> On Friday, 6 February 2015 22:20:58 UTC, Rob Gaddi wrote: > I found a recommendation at https://chriswarrick.com/blog/2014/09/15/ > python-apps-the-right-way-entry_points-and-scripts/ that seems to say > that I should get around this by simply having an empty __init__. I > guess I can move the program to foobar.py, have an empty __init__, a stub > __main__ that just calls foobar.main(), and an entry-point of > foobar.foobar:main . It just feels like a bit of a heavy solution to > have two unnecessary files. There's no real reason you have to take your code out of __init__.py, it's fine there. What it sounds like to have is a chunk of code that is basically the command line interface to your application. Presumably, if you've set things up for an entry point, that code is in a main() function. So the following should work: -- mypkg | -- __init__.py def main(): # whatever -- __main__.py import mypkg mypkg.main() Then define your entry point as calling mypkg:main This will give you an entry point script when you install your project, and the __main__.py file means that "python -m mypkg" will also work. If you want a working example, pip itself uses this structure (although it's a bit more complicated because pip's a bit bigger :-)) Hope this helps, Paul From ethan at stoneleaf.us Fri Feb 6 18:21:52 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 06 Feb 2015 15:21:52 -0800 Subject: Setuptools, __init__ and __main__ In-Reply-To: <85h9uyfy3v.fsf@benfinney.id.au> References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> Message-ID: <54D54C90.2070008@stoneleaf.us> On 02/06/2015 02:56 PM, Ben Finney wrote: > > It's not Setuptools per se, it's Python's import mechanism. It is a > deliberate design decision that direct import of a module makes that > module blind to its location in the package hierarchy. > > That's a design decision I deplore, because it makes something that > should be easy (write a command directly into a file and invoke that > file as an executable program) tortuously difficult when the program > comprises several modules. Can you explain that a bit more? Surely if the writing a command into a file was going to be easy, then so would be writing it to __main__.py instead? -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From steve+comp.lang.python at pearwood.info Fri Feb 6 18:49:40 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 07 Feb 2015 10:49:40 +1100 Subject: Monte Carlo probability calculation in Python References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> <51d12f89-7d03-41e9-a2c9-3960a15ed6aa@googlegroups.com> Message-ID: <54d55314$0$12979$c3e8da3$5496439d@news.astraweb.com> Paul Moore wrote: > On Thursday, 5 February 2015 23:02:42 UTC, Paul Moore wrote: >> Nice! Thanks both of you. I start to see how to think about these >> things now. I'll play around with some of the other examples I've got, >> and see how far I get. > > Just a quick status update, in case you're interested. With relatively > little work (considering I started not knowing much about numpy) I managed > to put together solutions for a couple of my friend's problems which ran > in basically the same time as his custom C code. Very impressive! Very nice! Care to share the code? -- Steven From steve+comp.lang.python at pearwood.info Fri Feb 6 18:56:35 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 07 Feb 2015 10:56:35 +1100 Subject: Setuptools, __init__ and __main__ References: <85lhkag1ts.fsf@benfinney.id.au> Message-ID: <54d554b4$0$13008$c3e8da3$5496439d@news.astraweb.com> Dave Angel wrote: > And don't name any source code __main__.py, That's incorrect. Starting from Python 2.6, __main__.py is reserved for the application main script in packages. That is, if you design your application as a package (not a single file) with this structure: myapplication/ +-- __init__.py +-- __main__.py +-- spam.py +-- eggs.py +-- cheese.py the __init__.py file is needed for it to be recognised as a package, and the __main__.py file is used when running the package. (Everything else is optional, of course, although it's hard to see why you would bother with a package instead of a single module if you weren't breaking the code up into multiple modules.) In other words, when you run `import myapplication` from within Python, the __init__.py module is imported. But when you do this from the shell: python -m myapplication Python will search the PYTHONPATH, find the myapplication package, and run myapplication/__main__.py as a script. Of course, you can also run __main__.py by hand: python /path/to/the/myapplication/__main__.py but having Python search the path is more convenient. > Finally, the mere presence of __init__.py has a special meaning, and > code in it should probably be limited to doing namespace munging that a > package may require to keep its public interface simpler. It's conventional to have an empty __init__.py, but not required. At the very least __init__.py should contain enough code (perhaps `from mypackage.spam import *`) to make `import mypackage` useful. -- Steven From davea at davea.name Fri Feb 6 19:17:18 2015 From: davea at davea.name (Dave Angel) Date: Fri, 06 Feb 2015 19:17:18 -0500 Subject: Setuptools, __init__ and __main__ In-Reply-To: <54d554b4$0$13008$c3e8da3$5496439d@news.astraweb.com> References: <85lhkag1ts.fsf@benfinney.id.au> <54d554b4$0$13008$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54D5598E.6010605@davea.name> On 02/06/2015 06:56 PM, Steven D'Aprano wrote: > Dave Angel wrote: > >> And don't name any source code __main__.py, > > > That's incorrect. > > Starting from Python 2.6, __main__.py is reserved for the application main > script in packages. That is, if you design your application as a package > (not a single file) with this structure: > > > myapplication/ > +-- __init__.py > +-- __main__.py > +-- spam.py > +-- eggs.py > +-- cheese.py > > > the __init__.py file is needed for it to be recognised as a package, and the > __main__.py file is used when running the package. (Everything else is > optional, of course, although it's hard to see why you would bother with a > package instead of a single module if you weren't breaking the code up into > multiple modules.) > > In other words, when you run `import myapplication` from within Python, the > __init__.py module is imported. But when you do this from the shell: > > python -m myapplication > > > Python will search the PYTHONPATH, find the myapplication package, and run > myapplication/__main__.py as a script. That's the main thing I just didn't know. I've seen __main__ from the perspective of the if __name__ == "__main__": where it's just an alias for the actual script name, and I didn't realize the duality. Sorry for the noise. > > Of course, you can also run __main__.py by hand: > > python /path/to/the/myapplication/__main__.py > > > but having Python search the path is more convenient. > > >> Finally, the mere presence of __init__.py has a special meaning, and >> code in it should probably be limited to doing namespace munging that a >> package may require to keep its public interface simpler. > > It's conventional to have an empty __init__.py, but not required. At the > very least __init__.py should contain enough code (perhaps `from > mypackage.spam import *`) to make `import mypackage` useful. > > > -- DaveA From ben+python at benfinney.id.au Fri Feb 6 19:44:47 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 07 Feb 2015 11:44:47 +1100 Subject: Incompatible idioms: relative imports, top-level program file (was: Setuptools, __init__ and __main__) References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> Message-ID: <858ugaft34.fsf_-_@benfinney.id.au> Ethan Furman writes: > On 02/06/2015 02:56 PM, Ben Finney wrote: > > It is a deliberate design decision that direct import of a module > > makes that module blind to its location in the package hierarchy. > > > > That's a design decision I deplore, because it makes something that > > should be easy (write a command directly into a file and invoke that > > file as an executable program) tortuously difficult when the program > > comprises several modules. > > Can you explain that a bit more? A program will often have enough complexity that its implementation occupies several sub-modules. There's no need to explose those in a site package, they normally only need to be local to the application. So the correct idiom is ?from __future__ import absolute_import? and keep the application's own implementation imported via relative imports in the same package: ?from . import implementation_detail?. But relative import is not possible when the top-level module is executed by the normal ?fooprog? usage. The top-level file is named ?fooprog? and is marked executable, so that it can be run directly as soon as it's written. Python deliberately divorces the top-level module from its package, so it can't access its own relative modules! The relative import fails with an ImportError ?Attempted relative import in non-package?. So the two correct idioms ? relative import for non-public modules, and run a file directly as a command ? are explicitly thwarted by Python's import behaviour when you try to use them together. > Surely if the writing a command into a file was going to be easy, then > so would be writing it to __main__.py instead? With Bash shell, or Perl, or Ruby, or countless other languages, I put the top-level code in the *exact same file* that I'm going to execute, named ?fooprog?. No additional infrastructure files unless my program architecture indicates they'll help. Python's refusal to allow this leads to roundabout hacks like ?make a sub-directory ?fooprog/?, write the code into ?__main__.py??, or ?run the program by typing ?python -m fooprog??, or ?write a ?./setup.py? configuration and dicker around with Distutils's rules and install it before you can run your program?. A system needing baroque hacks like this doesn't really deserve to be characterised as a ?scripting language?, IMO. Fortunately, I don't like that term anyway (we already have the term ?program? for what Python calls scripts) so avoid characterising Python that way for other reasons too :-) -- \ ?Sane people have an appropriate perspective on the relative | `\ importance of foodstuffs and human beings. Crazy people can't | _o__) tell the difference.? ?Paul Z. Myers, 2010-04-18 | Ben Finney From ethan at stoneleaf.us Fri Feb 6 20:12:44 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 06 Feb 2015 17:12:44 -0800 Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: <858ugaft34.fsf_-_@benfinney.id.au> References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> Message-ID: <54D5668C.9040600@stoneleaf.us> On 02/06/2015 04:44 PM, Ben Finney wrote: > Ethan Furman writes: > >> On 02/06/2015 02:56 PM, Ben Finney wrote: >>> It is a deliberate design decision that direct import of a module >>> makes that module blind to its location in the package hierarchy. >>> >>> That's a design decision I deplore, because it makes something that >>> should be easy (write a command directly into a file and invoke that >>> file as an executable program) tortuously difficult when the program >>> comprises several modules. >> >> Can you explain that a bit more? > > A program will often have enough complexity that its implementation > occupies several sub-modules. There's no need to explose those in a site > package, they normally only need to be local to the application. If they are not in the Python module path, how are they imported at all? > Python deliberately divorces the top-level module from its package, so > it can't access its own relative modules! The relative import fails with > an ImportError ?Attempted relative import in non-package?. My understanding is that imports is for libraries [1], and libraries must be in sys.path. Are you saying that after placing your top-level file's path in sys.path, that relative imports do not work? -- ~Ethan~ [1] which can be a single module -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ben+python at benfinney.id.au Fri Feb 6 20:50:46 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 07 Feb 2015 12:50:46 +1100 Subject: Incompatible idioms: relative imports, top-level program file References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> Message-ID: <85386ifq15.fsf@benfinney.id.au> Ethan Furman writes: > On 02/06/2015 04:44 PM, Ben Finney wrote: > > A program will often have enough complexity that its implementation > > occupies several sub-modules. There's no need to explose those in a > > site package, they normally only need to be local to the > > application. > > If they are not in the Python module path, how are they imported at > all? Only absolute imports use the module search path. The whole point of relative imports is to import a module within the same (or a sub-) package, without modifying the search path. > > Python deliberately divorces the top-level module from its package, > > so it can't access its own relative modules! > > My understanding is that imports is for libraries [1], and libraries > must be in sys.path. That's the arbitrary limitation I'm pointing out, yes. The program's private implementation modules should not need to be in the global module search path, merely to allow the top-level program to import them. It's contradictory to present relative imports with the justification they are for intra-package imports, and then cripple a top-level module so it can't access its own implementation modules via relative imports. -- \ ?Capitalism is the astounding belief that the most wickedest of | `\ men will do the most wickedest of things for the greatest good | _o__) of everyone.? ?John Maynard Keynes | Ben Finney From nad at acm.org Fri Feb 6 22:04:43 2015 From: nad at acm.org (Ned Deily) Date: Fri, 06 Feb 2015 19:04:43 -0800 Subject: Matplotlib import error References: Message-ID: In article , C Smith wrote: > I had python 2.7.6 installed on OS X yosemite, which has always worked > fine, until I tried to install matplotlib with pip. I got the same > error below and upgraded to 2.7.9, used pip to upgrade all the > packages, but still get the same error. > > >>> import matplotlib > Traceback (most recent call last): > File "", line 1, in > File "/Library/Python/2.7/site-packages/matplotlib/__init__.py", > line 180, in > from matplotlib.cbook import is_string_like > File "/Library/Python/2.7/site-packages/matplotlib/cbook.py", line > 33, in It looks like you have a mixture of packages, some left over from using the system Python 2.7 and, unfortunately, the system Python site-package directory is included at the sys.path search path for other Pythons, like the python.org Pythons. If you plan to just use the Python 2.7.9, go to /Library/Python/2.7/site-packages and rm everything there. Then use the 2.7.9 pip to install matplotlib. It should download and install (to /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack ages) the binary wheels for matplotlib and its dependencies, including numpy, and all just work. -- Ned Deily, nad at acm.org From rustompmody at gmail.com Fri Feb 6 22:13:25 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 6 Feb 2015 19:13:25 -0800 (PST) Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> Message-ID: <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> On Saturday, February 7, 2015 at 7:35:12 AM UTC+5:30, Ben Finney wrote: > Ethan Furman writes: > > > On 02/06/2015 04:44 PM, Ben Finney wrote: > > > A program will often have enough complexity that its implementation > > > occupies several sub-modules. There's no need to explose those in a > > > site package, they normally only need to be local to the > > > application. > > > > If they are not in the Python module path, how are they imported at > > all? > > Only absolute imports use the module search path. The whole point of > relative imports is to import a module within the same (or a sub-) > package, without modifying the search path. > > > > > > Python deliberately divorces the top-level module from its package, > > > so it can't access its own relative modules! > > > > My understanding is that imports is for libraries [1], and libraries > > must be in sys.path. > > That's the arbitrary limitation I'm pointing out, yes. The program's > private implementation modules should not need to be in the global > module search path, merely to allow the top-level program to import > them. > > It's contradictory to present relative imports with the justification > they are for intra-package imports, and then cripple a top-level module > so it can't access its own implementation modules via relative imports. Thanks Ben for these points. We techies need to always slide the rheostat along: 1. I am dumb 2. I am ignorant - need to read up some 3. Stuff is complex - more to read up 4. Stuff is complicated - after all the reading up, some general ideas may get clear but there remain all sorts of dark nooks and crannies 5. Stuff is a complete bloody mess - anyone who claims to understand, doesn't understand and doesn't understand that he doesn't understand. [Think Stroustrup saying something to the effect it takes a lifetime to learn C++ !] Your points and the earlier thread have helped me to slide the rheostat a couple of notches down! My analysis of the problem: Python docs are a model of clarity... mostly. They fail when the rubber hits the road - OS-specific stuff. In the attempt at keeping the docs generic, when things like how the 'insides' of python map to the 'outsides' - file-system-mapping - I find the docs are sorely inadequate. [You are of course free to slide my rheostat up :-) ] There is on the one hand python modules/packages mechanism with all the hell of dozens of incompatible versions of setuptools/distribute/distutils etc. On the other there is the OS-specific practices/policy such as https://www.debian.org/doc/packaging-manuals/python-policy/ And the dark unexplored territory in between. [Its a research project to figure out clashes between pip and apt] Hopefully python-devs will give up trying to be generic in this area and start documenting these things in an OS-specific way. Yeah I know it can be significantly more of a babel than just: Windows - This-a-way Linux - That-a-way eg: - Mismatch between say Fedora and Debian - Different 32-64 bit path-disambiguation on different windows systems - etc Still... we need to bite the bullet Since this has been somewhat of a rant I should say: Wherever comparable to python we may look, its just like this or worse. Perl/Ruby are probably slightly more mature Haskell is much worse - search for "cabal hell" Latex is probably about the same: eg a debian texlive bugreport of mine: https://lists.debian.org/debian-tex-maint/2014/06/msg00029.html followed by unhelpful/improper response of the debian-dev From mhr1224 at gmail.com Sat Feb 7 00:14:35 2015 From: mhr1224 at gmail.com (mhr1224 at gmail.com) Date: Fri, 6 Feb 2015 21:14:35 -0800 (PST) Subject: help for using Python rsa Message-ID: I use this code to create public and private keys: import rsa (pubkey, privkey) = rsa.newkeys(512) And then convert it to PEM format: exppub =pubkey.save_pkcs1(format='PEM') exppriv = privkey.save_pkcs1(format='PEM') When i encrypt massage with this keys: message = 'hello Bob!' crypto = rsa.encrypt(message, pubkey) encrypted massage is like this: "@\xc4\xb2\x14r\xf1x\xb8\xb2\t;\x9a:\x1dl\x11\xe2\x10\xa9E\xee\x8b\xac\xd0\xd3Y\xfb}\xd9@\xdd\x0c\xa5\xd2\xfc1\xd6\x06\xf0\xb8\x944\xe1\xc2r\xe5anyq\xac\xdfh\xeb\x10\x80\x98\xa1\xee'\xe6hpi" and i know it should be like this: SEcPB1mYNrfeE4zP4RI3z2K4Rh9HDNfPhuF28IyxHFjEOJ9Z+1zdIwPF0jsJGQDJyKpAju7dcYueHHXXeH8d+w== How can i change the format of this? From rustompmody at gmail.com Sat Feb 7 01:08:24 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 6 Feb 2015 22:08:24 -0800 (PST) Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> Message-ID: <320f55ef-7ea2-4e41-9cc8-94a68326ca35@googlegroups.com> On Saturday, February 7, 2015 at 8:43:44 AM UTC+5:30, Rustom Mody wrote: > There is on the one hand python modules/packages mechanism with all the > hell of dozens of incompatible versions of setuptools/distribute/distutils etc. > > On the other there is the OS-specific practices/policy such as > https://www.debian.org/doc/packaging-manuals/python-policy/ > > And the dark unexplored territory in between. > [Its a research project to figure out clashes between pip and apt] > Small example of this which I just tried. Wanted to try out sympy. apt-install promised ? GB download, ? GB space usage Just getting a src-tarball was: 6M download, 30M after opening the tar. Sure the full stuff with tex fonts and all may be pretty. But does not seem necessary to push it down everyone's throat unless needed/desired From davea at davea.name Sat Feb 7 01:47:38 2015 From: davea at davea.name (Dave Angel) Date: Sat, 07 Feb 2015 01:47:38 -0500 Subject: help for using Python rsa In-Reply-To: References: Message-ID: <54D5B50A.2070107@davea.name> On 02/07/2015 12:14 AM, mhr1224 at gmail.com wrote: > I use this code to create public and private keys: > import rsa > (pubkey, privkey) = rsa.newkeys(512) > > And then convert it to PEM format: > exppub =pubkey.save_pkcs1(format='PEM') > exppriv = privkey.save_pkcs1(format='PEM') > > When i encrypt massage with this keys: > message = 'hello Bob!' > crypto = rsa.encrypt(message, pubkey) > > encrypted massage is like this: > "@\xc4\xb2\x14r\xf1x\xb8\xb2\t;\x9a:\x1dl\x11\xe2\x10\xa9E\xee\x8b\xac\xd0\xd3Y\xfb}\xd9@\xdd\x0c\xa5\xd2\xfc1\xd6\x06\xf0\xb8\x944\xe1\xc2r\xe5anyq\xac\xdfh\xeb\x10\x80\x98\xa1\xee'\xe6hpi" > > and i know it should be like this: > SEcPB1mYNrfeE4zP4RI3z2K4Rh9HDNfPhuF28IyxHFjEOJ9Z+1zdIwPF0jsJGQDJyKpAju7dcYueHHXXeH8d+w== > > How can i change the format of this? > I can't help with the crypto, but I can help make it more likely that someone will be able to help. 1) Always specify the Python version and OS when you start a new thread. 2) When you show data, show how you're displaying that data. You have two strings, but the one is apparently a repr() of a string, while the other is not. And you don't specify how you "know it should be like this." Are you running some other encryption utility that gives you that message? Be specific. Something like: crypto = rsa.enc.... print( repr(crypto) ) displays as "@\xc4\xb2\x... but running mycrypto.exe key="xyzyzzyzyzy" data="hello Bob!" displays as SEcPB1mYNrf... Have you tried decrypting the encrypted message, using the privkey? Did it in fact produce the original? Could it be that the error is in how you're transporting the public key to your reference program? It might also be useful to show the types of each of the variables. For example, you probably cannot encrypt text, but only binary. So you might need to explicitly encode the text "hello Bob!" before encrypting it. -- DaveA From rosuav at gmail.com Sat Feb 7 02:15:27 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Feb 2015 18:15:27 +1100 Subject: help for using Python rsa In-Reply-To: References: Message-ID: On Sat, Feb 7, 2015 at 4:14 PM, wrote: > encrypted massage is like this: > "@\xc4\xb2\x14r\xf1x\xb8\xb2\t;\x9a:\x1dl\x11\xe2\x10\xa9E\xee\x8b\xac\xd0\xd3Y\xfb}\xd9@\xdd\x0c\xa5\xd2\xfc1\xd6\x06\xf0\xb8\x944\xe1\xc2r\xe5anyq\xac\xdfh\xeb\x10\x80\x98\xa1\xee'\xe6hpi" > > and i know it should be like this: > SEcPB1mYNrfeE4zP4RI3z2K4Rh9HDNfPhuF28IyxHFjEOJ9Z+1zdIwPF0jsJGQDJyKpAju7dcYueHHXXeH8d+w== > > How can i change the format of this? That looks like Base 64 encoded text, which you can easily produce from Python. But that isn't the base-64 encoded version of the above binary blob. Are you asking how to base-64 something, or are you specifically expecting that output? ChrisA From automn at willow.org Sat Feb 7 03:48:58 2015 From: automn at willow.org (Automn) Date: 07 Feb 2015 08:48:58 GMT Subject: logging question References: Message-ID: Hello, On 2014-08-13, Arulnambi Nandagoban wrote: > > My pc restarts regularly. Whenever it restarts I lose all the log since it > writes in the file once per day. Is there a way to log information in a > file as soon as it available. You can log in the /var/tmp directory instead of /tmp. The former does not get cleaned when you restart. You can rotate your /var/log logfile manually or with a cronjob or with a shell program. > > My application is a tcp server. Before I did it without using logger module, > I developed customized log module for the application. As soon as there is > a connection I append it in the log file. > > By that way I didn't lose any log information. > Try to find out why your logfile gets truncated. I don't know it very well in python but if you open a file with libc you get to choose if you truncate, AFAIK it can be in sys or logging module. HTH, Automn -- my blog : http://thediaryofelvishhealer.blogger.com/ Time heals. From wxjmfauth at gmail.com Sat Feb 7 04:04:50 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Sat, 7 Feb 2015 01:04:50 -0800 (PST) Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> Message-ID: <8bbe8567-4f5c-4fd4-ad8c-02e210c76120@googlegroups.com> Le samedi 7 f?vrier 2015 04:13:44 UTC+1, Rustom Mody a ?crit?: > On Saturday, February 7, 2015 at 7:35:12 AM UTC+5:30, Ben Finney wrote: > > Ethan Furman writes: > > > > > On 02/06/2015 04:44 PM, Ben Finney wrote: > > > > A program will often have enough complexity that its implementation > > > > occupies several sub-modules. There's no need to explose those in a > > > > site package, they normally only need to be local to the > > > > application. > > > > > > If they are not in the Python module path, how are they imported at > > > all? > > > > Only absolute imports use the module search path. The whole point of > > relative imports is to import a module within the same (or a sub-) > > package, without modifying the search path. > > > > > > > > > > Python deliberately divorces the top-level module from its package, > > > > so it can't access its own relative modules! > > > > > > My understanding is that imports is for libraries [1], and libraries > > > must be in sys.path. > > > > That's the arbitrary limitation I'm pointing out, yes. The program's > > private implementation modules should not need to be in the global > > module search path, merely to allow the top-level program to import > > them. > > > > It's contradictory to present relative imports with the justification > > they are for intra-package imports, and then cripple a top-level module > > so it can't access its own implementation modules via relative imports. > > Thanks Ben for these points. > > We techies need to always slide the rheostat along: > > 1. I am dumb > 2. I am ignorant - need to read up some > 3. Stuff is complex - more to read up > 4. Stuff is complicated - after all the reading up, some general ideas may > get clear but there remain all sorts of dark nooks and crannies > 5. Stuff is a complete bloody mess - anyone who claims to understand, doesn't understand and doesn't understand that he doesn't understand. > [Think Stroustrup saying something to the effect it takes a lifetime to learn C++ !] > > Your points and the earlier thread have helped me to slide the rheostat a > couple of notches down! > > My analysis of the problem: > > Python docs are a model of clarity... mostly. > They fail when the rubber hits the road - OS-specific stuff. > > In the attempt at keeping the docs generic, when things like how the 'insides' > of python map to the 'outsides' - file-system-mapping - I find the docs are sorely inadequate. [You are of course free to slide my rheostat up :-) ] > > There is on the one hand python modules/packages mechanism with all the > hell of dozens of incompatible versions of setuptools/distribute/distutils etc. > > On the other there is the OS-specific practices/policy such as > https://www.debian.org/doc/packaging-manuals/python-policy/ > > And the dark unexplored territory in between. > [Its a research project to figure out clashes between pip and apt] > > Hopefully python-devs will give up trying to be generic in this area and start > documenting these things in an OS-specific way. > > Yeah I know it can be significantly more of a babel than just: > Windows - This-a-way > Linux - That-a-way > > eg: > - Mismatch between say Fedora and Debian > - Different 32-64 bit path-disambiguation on different windows systems > - etc > > Still... we need to bite the bullet > > Since this has been somewhat of a rant I should say: > Wherever comparable to python we may look, its just like this or worse. > > Perl/Ruby are probably slightly more mature > Haskell is much worse - search for "cabal hell" > Latex is probably about the same: eg a debian texlive bugreport of mine: > https://lists.debian.org/debian-tex-maint/2014/06/msg00029.html > followed by unhelpful/improper response of the debian-dev I'm still scratching my head on how to install ipython. For me it's blackbox. I tested, reported, worked on and with all wxPython versions during years (from ver 2.0 up to 2.9). Now, I'm unable to install wxPhoenix. From sturla.molden at gmail.com Sat Feb 7 07:30:28 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Sat, 7 Feb 2015 12:30:28 +0000 (UTC) Subject: Monte Carlo probability calculation in Python References: <63f603bc-431c-45e6-a496-0a28132f99d6@googlegroups.com> Message-ID: <2010385682445004448.388952sturla.molden-gmail.com@news.gmane.org> Paul Moore wrote: > > Yes. And a number of other variations. None gave anything that seemed to > relate. It's quite likely though that I'm simply not understanding how > things like pymc (which came up in the searches) might help me, or how to > convert my problem into a Monte Carlo integration problem (another topic > that came up a lot, for example). So if there are specific links from > such a search that match well to the problem as I described it above, I'd > be really grateful for pointers. PyMC and emcee are used for solving complicated integrals that often turn up in Bayesian statistics. They have the same usecase as the R-like application BUGS (WinBUGS). If you don't know what Markov Chain Monte Carlo, Gibbs sampler or Metropolis-Hastings means you probably don't want this. It is rather hardcore numerical mathematics for solving a particular problem (multidimensional integrals that are not analytically tractable), not something you would use for any kind of Monte Carlo simulation. Sturla From jessemuzziddu at mail.com Sat Feb 7 07:51:54 2015 From: jessemuzziddu at mail.com (MICHELE RAGAZZI. ODEY GIANO.) Date: Sat, 7 Feb 2015 04:51:54 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From jessemuzziddu at mail.com Sat Feb 7 07:52:47 2015 From: jessemuzziddu at mail.com (MICHELE RAGAZZI. ODEY GIANO.) Date: Sat, 7 Feb 2015 04:52:47 -0800 (PST) Subject: -- redacted -- In-Reply-To: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> References: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> Message-ID: -- redacted -- From illusiontechniques at gmail.com Sat Feb 7 10:23:30 2015 From: illusiontechniques at gmail.com (C Smith) Date: Sat, 7 Feb 2015 07:23:30 -0800 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On Fri, Feb 6, 2015 at 7:04 PM, Ned Deily wrote: > In article > , > C Smith wrote: >> I had python 2.7.6 installed on OS X yosemite, which has always worked >> fine, until I tried to install matplotlib with pip. I got the same >> error below and upgraded to 2.7.9, used pip to upgrade all the >> packages, but still get the same error. >> >> >>> import matplotlib >> Traceback (most recent call last): >> File "", line 1, in >> File "/Library/Python/2.7/site-packages/matplotlib/__init__.py", >> line 180, in >> from matplotlib.cbook import is_string_like >> File "/Library/Python/2.7/site-packages/matplotlib/cbook.py", line >> 33, in > > It looks like you have a mixture of packages, some left over from using > the system Python 2.7 and, unfortunately, the system Python site-package > directory is included at the sys.path search path for other Pythons, > like the python.org Pythons. If you plan to just use the Python 2.7.9, > go to /Library/Python/2.7/site-packages and rm everything there. Then > use the 2.7.9 pip to install matplotlib. It should download and install > (to > /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack > ages) the binary wheels for matplotlib and its dependencies, including > numpy, and all just work. > > -- > Ned Deily, > nad at acm.org > > -- > https://mail.python.org/mailman/listinfo/python-list A lot of the stuff in that directory took a long time to get compiled and working properly. Buildozer, pcapy, scapy are more important than matplotlib to me, and they work well. From breamoreboy at yahoo.co.uk Sat Feb 7 10:36:25 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 07 Feb 2015 15:36:25 +0000 Subject: Matplotlib import error In-Reply-To: References: Message-ID: On 07/02/2015 15:23, C Smith wrote: > On Fri, Feb 6, 2015 at 7:04 PM, Ned Deily wrote: >> In article >> , >> C Smith wrote: >>> I had python 2.7.6 installed on OS X yosemite, which has always worked >>> fine, until I tried to install matplotlib with pip. I got the same >>> error below and upgraded to 2.7.9, used pip to upgrade all the >>> packages, but still get the same error. >>> >>>>>> import matplotlib >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "/Library/Python/2.7/site-packages/matplotlib/__init__.py", >>> line 180, in >>> from matplotlib.cbook import is_string_like >>> File "/Library/Python/2.7/site-packages/matplotlib/cbook.py", line >>> 33, in >> >> It looks like you have a mixture of packages, some left over from using >> the system Python 2.7 and, unfortunately, the system Python site-package >> directory is included at the sys.path search path for other Pythons, >> like the python.org Pythons. If you plan to just use the Python 2.7.9, >> go to /Library/Python/2.7/site-packages and rm everything there. Then >> use the 2.7.9 pip to install matplotlib. It should download and install >> (to >> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack >> ages) the binary wheels for matplotlib and its dependencies, including >> numpy, and all just work. >> >> -- >> Ned Deily, >> nad at acm.org >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > A lot of the stuff in that directory took a long time to get compiled > and working properly. Buildozer, pcapy, scapy are more important than > matplotlib to me, and they work well. > An alternative is to use another OS that perhaps doesn't run on overrated, overpriced hardware from an overrated company. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From Seymore4Head at Hotmail.invalid Sat Feb 7 11:50:30 2015 From: Seymore4Head at Hotmail.invalid (Seymore4Head) Date: Sat, 07 Feb 2015 11:50:30 -0500 Subject: How to extract a movie title out of a text file Message-ID: What I would like to be able to do is download and save to a folder all the srr files in this Usenet group. alt.binaries.moovee I would then like to be able to search for a title in the text file. Anyone care to come up with the Python code to do this? From none at mailinator.com Sat Feb 7 12:25:29 2015 From: none at mailinator.com (mm0fmf) Date: Sat, 07 Feb 2015 17:25:29 +0000 Subject: How to extract a movie title out of a text file In-Reply-To: References: Message-ID: On 07/02/2015 16:50, Seymore4Head wrote: > Anyone care to come up with the Python code to do this? > No problem, where I work charges me out at $2250/day plus expenses etc. I'll do it for half of that So send me $562 upfront and I'll get on with this for you. 24/7 hotline support is included in the price for the 1st year. From breamoreboy at yahoo.co.uk Sat Feb 7 12:36:06 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 07 Feb 2015 17:36:06 +0000 Subject: How to extract a movie title out of a text file In-Reply-To: References: Message-ID: On 07/02/2015 16:50, Seymore4Head wrote: > What I would like to be able to do is download and save to a folder > all the srr files in this Usenet group. alt.binaries.moovee > > I would then like to be able to search for a title in the text file. > > Anyone care to come up with the Python code to do this? > Sure, just write out a cheque made payable to the Python Software Foundation, we'll fill in the details and when it's been cashed we'll provide the code. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From fomcl at yahoo.com Sat Feb 7 12:59:48 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sat, 7 Feb 2015 09:59:48 -0800 Subject: How to extract a movie title out of a text file Message-ID: <1423331988.92006.BPMail_high_carrier@web163805.mail.gq1.yahoo.com> ----------------------------- On Sat, Feb 7, 2015 6:36 PM CET Mark Lawrence wrote: >On 07/02/2015 16:50, Seymore4Head wrote: >> What I would like to be able to do is download and save to a folder >> all the srr files in this Usenet group. alt.binaries.moovee >> >> I would then like to be able to search for a title in the text file. >> >> Anyone care to come up with the Python code to do this? >> > >Sure, just write out a cheque made payable to the Python Software Foundation, we'll fill in the details and when it's been cashed we'll provide the code. +1 for that remark >-- My fellow Pythonistas, ask not what our language can do for you, ask >what you can do for our language. > >Mark Lawrence > >-- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Sat Feb 7 15:31:43 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 07:31:43 +1100 Subject: How to extract a movie title out of a text file In-Reply-To: References: Message-ID: On Sun, Feb 8, 2015 at 3:50 AM, Seymore4Head wrote: > What I would like to be able to do is download and save to a folder > all the srr files in this Usenet group. alt.binaries.moovee > > I would then like to be able to search for a title in the text file. > > Anyone care to come up with the Python code to do this? If you mean .srt files, I don't think they have movie titles... but they're plain text, so all you'd need to do is search for the content you're after. ChrisA From xpegenaute at telepolis.es Sat Feb 7 17:09:02 2015 From: xpegenaute at telepolis.es (Xavier Pegenaute) Date: Sat, 07 Feb 2015 22:09:02 +0000 Subject: How to Mock a mongodb Message-ID: <54D68CFE.5070904@telepolis.es> Dear, I am trying to mock the use of a mongo db and I am having some trouble. Appears that I am not able to return a desired value from mongo.find().count(). I made a proof of concept to try to reduce complexity of the real problem. You can find the code which is going to be tested in [1], and the code of the test case in [2]. Do you know exactly wat's wrong with it?, or may you point me to some direction? Thanks, Xavi [1] - http://pastebin.com/0vGqsURk [2] - http://pastebin.com/6Y6K4dUn From xpegenaute at poblenousensefils.net Sat Feb 7 17:09:10 2015 From: xpegenaute at poblenousensefils.net (Xavier Pegenaute) Date: Sat, 07 Feb 2015 22:09:10 +0000 Subject: How to Mock a mongodb Message-ID: <54D68D06.70807@poblenousensefils.net> Dear, I am trying to mock the use of a mongo db and I am having some trouble. Appears that I am not able to return a desired value from mongo.find().count(). I made a proof of concept to try to reduce complexity of the real problem. You can find the code which is going to be tested in [1], and the code of the test case in [2]. Do you know exactly wat's wrong with it?, or may you point me to some direction? Thanks, Xavi [1] - http://pastebin.com/0vGqsURk [2] - http://pastebin.com/6Y6K4dUn From greg.ewing at canterbury.ac.nz Sat Feb 7 17:22:04 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 08 Feb 2015 11:22:04 +1300 Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: <320f55ef-7ea2-4e41-9cc8-94a68326ca35@googlegroups.com> References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> <320f55ef-7ea2-4e41-9cc8-94a68326ca35@googlegroups.com> Message-ID: Rustom Mody wrote: > Wanted to try out sympy. > apt-install promised ? GB download, ? GB space usage > > Just getting a src-tarball was: 6M download, 30M after opening the tar. Have you actually tried compiling and using that tarball, though? Sympy hooks into a lot of other libraries that are themselves quite large. Apt-get was probably planning to download and install all of them, hence the large download size. If you tried to install from source, you would likely have to install all those dependencies yourself, or configure it to skip the ones you don't want if it allows that. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 7 17:28:11 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 08 Feb 2015 11:28:11 +1300 Subject: How to extract a movie title out of a text file In-Reply-To: References: Message-ID: Seymore4Head wrote: > What I would like to be able to do is download and save to a folder > all the srr files in this Usenet group. alt.binaries.moovee The nntplib module in the standard library would be a good place to start. -- Greg From none at mailinator.com Sat Feb 7 17:45:02 2015 From: none at mailinator.com (mm0fmf) Date: Sat, 07 Feb 2015 22:45:02 +0000 Subject: How to Mock a mongodb In-Reply-To: References: Message-ID: On 07/02/2015 22:09, Xavier Pegenaute wrote: > Dear, > > I am trying to mock the use of a mongo db and I am having some trouble. > Appears that I am not able to return a desired value from > mongo.find().count(). I made a proof of concept to try to reduce > complexity of the real problem. > > You can find the code which is going to be tested in [1], and the code > of the test case in [2]. > > Do you know exactly wat's wrong with it?, or may you point me to some > direction? > > Thanks, > Xavi > [1] - http://pastebin.com/0vGqsURk > [2] - http://pastebin.com/6Y6K4dUn Xavi, the majority of people in this group are not going to click a link as a matter of principle. So you need to paste some real code and then you'll find there will be people who will offer you really good help. In fact this group seems to contain an more clued up and helpful people than I can ever recall in any other newsgroup. Post a simple 'reproducer' of your issue and you should get some help. Andy From albert at spenarnc.xs4all.nl Sat Feb 7 18:35:03 2015 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 07 Feb 2015 23:35:03 GMT Subject: What killed Smalltalk could kill Python References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d6a127$0$6901$e4fe514c@dreader36.news.xs4all.nl> In article , Tim Chase wrote: >On 2015-01-22 03:34, Steven D'Aprano wrote: >> In 2009, Robert Martin gave a talk at RailsConf titled "What Killed >> Smalltalk Could Kill Ruby". > >Holy pacing, Batman. Watching it at 2x leaves me wondering how much >of the stage was worn off during the presentation. > >> And now it's all but dead. Why did it die, and how can Python (or >> Ruby for that matter) avoid the same fate? > >In my experience, most Python has a particularly low WTF-per-minute >score. > >But mostly Michael's reply addresses my biggest pain points the last >couple times I tried Smalltalk: The whole "images" thing impeded me >from easily using my development preferred environment. > >With Python, I can just install it and then either fire up the >REPL, or type some code into a file and run it (same I suppose would >go for Ruby). > >I fought for over an hour trying to figure out how to just get >ANYTHING to run in Smalltalk. I installed Squeak on Debian and yet I >couldn't get any code examples to run. I had to go find some >environments on the web, download them, modify them, and eventually >something ran. Eventually I just gave up and returned to a world >where everything made sense. This is the introductory chapter of my Forth: " 4 Manual ******** 4.1 Getting started =================== 4.1.1 Hello world! ------------------ Type `lina64' to get into your interactive Forth system. You will see a signon message. While sitting in your interactive Forth doing a "hello world" is easy: "Hello world!" TYPE Hello world! OK Note that the computer ends its output with `OK' to indicate that it has completed the command. Making it into an interactively usable program is also easy: : HELLO "Hello world!" TYPE CR ; OK HELLO Hello world! OK This means you type the command `HELLO' while you are in lina64. As soon as you leave lina64, the new command is gone. If you want to use the program a second time, you can put it in a file `hello.frt'. It just contains the definition we typed earlier: : HELLO "Hello world!" TYPE CR ; This file can be `INCLUDED' inorder to add the command `HELLO' to your Forth environment, like so: "hello.frt" INCLUDED OK HELLO Hello world! OK During development you probably have started with `lina64 -e', so you need just type INCLUDE hello.frt In order to make a stand alone program to say hello you can use that same source file, again `hello.frt'. Now build the program by lina64 -c hello.frt (That is `c' for compile.) The result is a file `hello' . This file can be run from your command interpreter, or shell. It is a single file that you can pass to some one else to run on their computer, without the need for them to install Forth. For the compiler to run you must have the library correctly installed. " Seems like I did it slightly better. (Mind you, this is chapter 4, for beginners there is chapter 2, e.g. if the `` : '' word puzzles you.) Groetjes Albert > >-tkc > > -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From albert at spenarnc.xs4all.nl Sat Feb 7 18:54:33 2015 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 07 Feb 2015 23:54:33 GMT Subject: What killed Smalltalk could kill Python References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> <87r3un8r0y.fsf@jester.gateway.sonic.net> Message-ID: <54d6a5b9$0$6901$e4fe514c@dreader36.news.xs4all.nl> In article , Chris Angelico wrote: >On Thu, Jan 22, 2015 at 1:53 PM, Paul Rubin wrote: >>> If someone's unfazed by the "it'll take you years before you can >>> actually write a saleable game" consideration, >> >> Wanting to write games is a completely different topic than wanting to >> sell them. It's just like any other creative outlet. Most people who >> teach themselves to juggle do it because juggling is fun, not because >> they want to join the circus. > >True, but even a playable game is a long way beyond a first-day >programmer. (By "playable" I mean something that someone other than >its author would play and enjoy.) It's fine as a goal, but needs to be >viewed with a perspective of "that's where I'm trying to get to", not >"now I'm going to start writing games". Not to mention that mostly a game is understood, not as something like chess, but an FPS (first person shooter) game. But that is real time programming, one league beyond beginners procedural (sequential) or functional programming. The result is either a disappointment or the illusion of having created something while in fact one used a frame work where all the hard work has been done. > >ChrisA Groetjes Albert -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From albert at spenarnc.xs4all.nl Sat Feb 7 19:45:20 2015 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 08 Feb 2015 00:45:20 GMT Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> In article , Ethan Furman wrote: >-=-=-=-=-=- > >On 01/21/2015 08:30 PM, Steven D'Aprano wrote: >> >> So what is this unspeakable, nightmarish, cryptic abomination going to look >> like? Here's an example from PEP 484: >> >> def greeting(name: str) -> str: >> return 'Hello ' + name >> >> >> I don't know about you, but I think anyone who cannot read that and intuit >> that argument `name` is a string and the return result is also a string > >There is nothing inherently intuitive about that syntax. The : makes it >look like a dictionary (but it isn't) and the >-> looks like a pointer to something (but it isn't). It is too bad `` -> '' as a token is now taken. I wanted to propose to replace the ternary syntax lambda .. : .. by a regular operator .. -> .. then we could have x -> x**2 instead of lambda x : x**2 Moreover the value of a function would be a lambda not def square(x): x**2 but square = x->x**2 or mult = x,y -> result = 0 for i in range(x): result +=y return result doing away with the ternary operator def def .. ( .. ) : .. replacing it by two binary operators, one of them (=) being thoroughly familiar. It is horrifying to see that def is now becoming a quaternary operator def .. ( .. ) --> .. : .. Also name:str is the wrong order. I would propose to use :: to prevent confusion. Then I would allow :: in front of all objects everywhere to trigger a warning if at that point the objects is not of the right type. I think it is quite natural that float: sets the expectation that a float is coming. float:: x = get_some_crap_from_an obscure_windows_box( a, B, c, a_lighter, some_list, REAL_ISB_MASK ) > >> is probably going to have bigger troubles with Python than just type-hinting. > >Yup, true -- I do find writing meta-classes takes extra work. ;) > >-- >~Ethan~ > > > >-=-=-=-=-=- >[Attachment type=application/pgp-signature, name=signature.asc] >-=-=-=-=-=- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From rosuav at gmail.com Sat Feb 7 19:57:01 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 11:57:01 +1100 Subject: What killed Smalltalk could kill Python In-Reply-To: <54d6a5b9$0$6901$e4fe514c@dreader36.news.xs4all.nl> References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> <87r3un8r0y.fsf@jester.gateway.sonic.net> <54d6a5b9$0$6901$e4fe514c@dreader36.news.xs4all.nl> Message-ID: On Sun, Feb 8, 2015 at 10:54 AM, Albert van der Horst wrote: > Not to mention that mostly a game is understood, not as something like > chess, but an FPS (first person shooter) game. > But that is real time programming, one league beyond beginners > procedural (sequential) or functional programming. > The result is either a disappointment or the illusion of having created > something while in fact one used a frame work where all the hard work > has been done. Even worse, it's usually graphical. From what I've heard from companies and people that produce graphical games, there's at least as much work in creating assets (images, 3D models, textures, stuff) as there is in writing code. And that's before you even consider writing a storyline, which is something that I wouldn't expect a new programmer to worry too much about ("just go around shooting stuff up" makes a fine storyline), but which the best commercial games always put a lot of work into. (And it does improve the game significantly. I wouldn't have spent anything like as many hours on Alice: Madness Returns as I have if it didn't have the storyline it does.) If someone's going to create a game from scratch, it should probably be a puzzle game. Things like 2048 wouldn't take nearly as much effort as an FPS. Turn-based rather than real-time, no heavy graphics, a simple bit of randomness and only a few controls. Pretty straight-forward. ChrisA From rosuav at gmail.com Sat Feb 7 20:01:04 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 12:01:04 +1100 Subject: Python is DOOMED! Again! In-Reply-To: <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> Message-ID: On Sun, Feb 8, 2015 at 11:45 AM, Albert van der Horst wrote: > def square(x): x**2 > but > square = x->x**2 > > or > > mult = x,y -> > result = 0 > for i in range(x): > result +=y > return result > > doing away with the ternary operator def > > def .. ( .. ) : .. > > replacing it by two binary operators, one of them (=) being thoroughly familiar. Thing is, "def" isn't just assignment. It also takes the name and stores it in the object. There's a distinct and visible difference between: def square(x): return x**2 and def func(x): return x**2 square = func So if you were to use your alternate syntax everywhere, you'd basically be throwing away all the benefits of def over lambda. Now, you may well be able to justify and implement an alternative for lambda that works this way. (And it's probably already been done, too. Maybe using the actual symbol ? rather than ->.) But I don't think you'll get anywhere with def. ChrisA From cs at zip.com.au Sat Feb 7 21:55:37 2015 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 8 Feb 2015 13:55:37 +1100 Subject: help for using Python rsa In-Reply-To: References: Message-ID: <20150208025537.GA99535@cskk.homeip.net> On 07Feb2015 18:15, Chris Angelico wrote: >On Sat, Feb 7, 2015 at 4:14 PM, wrote: >> encrypted massage is like this: >> "@\xc4\xb2\x14r\xf1x\xb8\xb2\t;\x9a:\x1dl\x11\xe2\x10\xa9E\xee\x8b\xac\xd0\xd3Y\xfb}\xd9@\xdd\x0c\xa5\xd2\xfc1\xd6\x06\xf0\xb8\x944\xe1\xc2r\xe5anyq\xac\xdfh\xeb\x10\x80\x98\xa1\xee'\xe6hpi" >> >> and i know it should be like this: >> SEcPB1mYNrfeE4zP4RI3z2K4Rh9HDNfPhuF28IyxHFjEOJ9Z+1zdIwPF0jsJGQDJyKpAju7dcYueHHXXeH8d+w== >> >> How can i change the format of this? > >That looks like Base 64 encoded text, which you can easily produce >from Python. But that isn't the base-64 encoded version of the above >binary blob. Indeed: >>> base64.b64decode('SEcPB1mYNrfeE4zP4RI3z2K4Rh9HDNfPhuF28IyxHFjEOJ9Z+1zdIwPF0jsJGQDJyKpAju7dcYueHHXXeH8d+w==') 'HG\x0f\x07Y\x986\xb7\xde\x13\x8c\xcf\xe1\x127\xcfb\xb8F\x1fG\x0c\xd7\xcf\x86\xe1v\xf0\x8c\xb1\x1cX\xc48\x9fY\xfb\\\xdd#\x03\xc5\xd2;\t\x19\x00\xc9\xc8\xaa@\x8e\xee\xddq\x8b\x9e\x1cu\xd7x\x7f\x1d\xfb' But as you say, it is base64 compatible. Maybe the 'SEcP...' string is base64 encoding of some encapsulation of the encrypted string? >Are you asking how to base-64 something, or are you specifically >expecting that output? Given that Dave Angel has a tool that displays the 'SEcP...' string using mhr1224's input string, I'd say we're missing some context. Cheers, Cameron Simpson You my man are a danger to society and should be taken out of society for all our sakes. As to what is done to you once removed I couldn't care less. - Roy G. Culley, Unix Systems Administrator From rosuav at gmail.com Sat Feb 7 22:00:29 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 14:00:29 +1100 Subject: help for using Python rsa In-Reply-To: <20150208025537.GA99535@cskk.homeip.net> References: <20150208025537.GA99535@cskk.homeip.net> Message-ID: On Sun, Feb 8, 2015 at 1:55 PM, Cameron Simpson wrote: > Given that Dave Angel has a tool that displays the 'SEcP...' string using > mhr1224's input string, I'd say we're missing some context. I don't think he does; that was simulated output as part of an example of "here's how to ask your question". ChrisA From d at davea.name Sat Feb 7 22:21:50 2015 From: d at davea.name (Dave Angel) Date: Sat, 07 Feb 2015 22:21:50 -0500 Subject: help for using Python rsa In-Reply-To: References: <20150208025537.GA99535@cskk.homeip.net> Message-ID: <54D6D64E.4090809@davea.name> On 02/07/2015 10:00 PM, Chris Angelico wrote: > On Sun, Feb 8, 2015 at 1:55 PM, Cameron Simpson wrote: >> Given that Dave Angel has a tool that displays the 'SEcP...' string using >> mhr1224's input string, I'd say we're missing some context. > > I don't think he does; that was simulated output as part of an example > of "here's how to ask your question". Precisely. We're just guessing till the OP clarifies things. And I'm beginning to wonder if he plans to. -- DaveA From steve+comp.lang.python at pearwood.info Sun Feb 8 02:55:43 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 08 Feb 2015 18:55:43 +1100 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> Message-ID: <54d71681$0$13000$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Sun, Feb 8, 2015 at 11:45 AM, Albert van der Horst > wrote: >> def square(x): x**2 >> but >> square = x->x**2 >> >> or >> >> mult = x,y -> >> result = 0 >> for i in range(x): >> result +=y >> return result >> >> doing away with the ternary operator def >> >> def .. ( .. ) : .. >> >> replacing it by two binary operators, one of them (=) being thoroughly >> familiar. > > Thing is, "def" isn't just assignment. It also takes the name and > stores it in the object. There's a distinct and visible difference > between: > > def square(x): return x**2 > > and > > def func(x): return x**2 > square = func > > So if you were to use your alternate syntax everywhere, you'd > basically be throwing away all the benefits of def over lambda. If this were syntax, then the compiler could just as easily set the function name from -> as from def. Lambda has the limitations that it has because it is an expression, not because of magical "def" properties. I think it is a total waste of good syntax to limit a hypothetical -> operator to a mere replacement for "def". It would be much more interesting for pattern-matching, rather like Haskell: fact 0 = 1 fact n = n*fact(n-1) That is more or less equivalent to pseudocode: def fact(arg): if arg == 0: return 1 if arg matches n: return n*fact(n-1) What does it mean to "match n"? Haskell can infer that n must be an integer. (It might even be able to infer that n must be a *positive* integer. I'd test it myself except I'm lazy.) So if it receives an integer argument which isn't 0, it will match n. Because indentation in Python is significant, we could drop the repeated use of the function name by indenting the individual patterns, and use type annotations to specify different types: fact 0 -> 1 n:int -> n*fact(n-1) x:float -> math.gamma(x-1) As I said, this is not a thought-out proposal, just some ideal musings, but it seems a crying shame to waste a potential pattern match operator for a mere abbreviation to "def". -- Steven From steve+comp.lang.python at pearwood.info Sun Feb 8 02:59:50 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 08 Feb 2015 18:59:50 +1100 Subject: What killed Smalltalk could kill Python References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> <87r3un8r0y.fsf@jester.gateway.sonic.net> <54d6a5b9$0$6901$e4fe514c@dreader36.news.xs4all.nl> Message-ID: <54d71776$0$13000$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Sun, Feb 8, 2015 at 10:54 AM, Albert van der Horst > wrote: >> Not to mention that mostly a game is understood, not as something like >> chess, but an FPS (first person shooter) game. >> But that is real time programming, one league beyond beginners >> procedural (sequential) or functional programming. >> The result is either a disappointment or the illusion of having created >> something while in fact one used a frame work where all the hard work >> has been done. > > Even worse, it's usually graphical. From what I've heard from > companies and people that produce graphical games, there's at least as > much work in creating assets (images, 3D models, textures, stuff) as > there is in writing code. And that's before you even consider writing > a storyline, which is something that I wouldn't expect a new > programmer to worry too much about ("just go around shooting stuff up" > makes a fine storyline), but which the best commercial games always > put a lot of work into. (And it does improve the game significantly. I > wouldn't have spent anything like as many hours on Alice: Madness > Returns as I have if it didn't have the storyline it does.) If a professional games company has their coders writing the plot and designing the graphics, they deserve to fail. (Well, that's a bit harsh... there's still room in the world for small indy companies, and even one-person projects.) You hire artists to design your artwork, and writers to write your story, and programmers to program your code. It is rare to have one person able to do all three to professional quality. -- Steven From rosuav at gmail.com Sun Feb 8 03:21:07 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 19:21:07 +1100 Subject: Python is DOOMED! Again! In-Reply-To: <54d71681$0$13000$c3e8da3$5496439d@news.astraweb.com> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> <54d71681$0$13000$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 8, 2015 at 6:55 PM, Steven D'Aprano wrote: > If this were syntax, then the compiler could just as easily set the function > name from -> as from def. Lambda has the limitations that it has because it > is an expression, not because of magical "def" properties. True, it could, but it would be odd that what looks like assignment and an expression is actually magical syntax. But it might be nice to have something that functions as lambda currently does, unless it detects that it's being assigned directly to a name, in which case the magic kicks in and it gets a name. square = x->x**2 # Name is "square" operator["square"] = x->x**2 # Maybe? create_operator("square", x->x**2) # Name is "" But I suspect that this would create some hairy grammatical quirks. Still, as an "alternate syntax for creating a lambda function", it seems at least plausible. ChrisA From rosuav at gmail.com Sun Feb 8 03:24:15 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Feb 2015 19:24:15 +1100 Subject: What killed Smalltalk could kill Python In-Reply-To: <54d71776$0$13000$c3e8da3$5496439d@news.astraweb.com> References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> <87r3un8r0y.fsf@jester.gateway.sonic.net> <54d6a5b9$0$6901$e4fe514c@dreader36.news.xs4all.nl> <54d71776$0$13000$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 8, 2015 at 6:59 PM, Steven D'Aprano wrote: > If a professional games company has their coders writing the plot and > designing the graphics, they deserve to fail. (Well, that's a bit harsh... > there's still room in the world for small indy companies, and even > one-person projects.) You hire artists to design your artwork, and writers > to write your story, and programmers to program your code. It is rare to > have one person able to do all three to professional quality. Of course you do... in a pro company that can afford to have that many people. As you say, some small companies double up; and this discussion was in the context of someone saying "I want to learn computer programming so I can write games", which usually implies a one-person project. But if anything, that makes the point even stronger: you do NOT want to start in as a programmer on a project that'll require a lot more than just programming skill. ChrisA From ian.g.kelly at gmail.com Sun Feb 8 03:31:48 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 8 Feb 2015 01:31:48 -0700 Subject: Python is DOOMED! Again! In-Reply-To: <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> Message-ID: On Sat, Feb 7, 2015 at 5:45 PM, Albert van der Horst wrote: > It is too bad `` -> '' as a token is now taken. > I wanted to propose to replace the ternary syntax > lambda .. : .. > by a regular operator > .. -> .. > then we could have > x -> x**2 > instead of > lambda x : x**2 Well, I don't think the existing syntax is incompatible with your proposal. As it is, the -> token can only appear after the argument list of a def statement, so there would be no grammatical ambiguity. I do think that such a proposal is unlikely to gain wide support though. > Moreover the value of a function would be a lambda > > not > def square(x): x**2 > but > square = x->x**2 This would be an anti-pattern. The def statement associates the name "square" with the function's __name__ attribute, which is useful for debugging and introspection. The proposed assignment statement does not. > mult = x,y -> > result = 0 > for i in range(x): > result +=y > return result I don't like this at all. I read "x -> x**2" as denoting a mapping from a bound variable to an expression. A whole function body just feels wrong here. > doing away with the ternary operator def def is a statement, not an operator. > replacing it by two binary operators, one of them (=) being thoroughly familiar. = is also not an operator. > Also name:str is the wrong order. I disagree; "name: type" is linguistically correct, with the colon denoting that what comes after describes what comes before. Without the colon, the opposite order would make more sense. From marko at pacujo.net Sun Feb 8 05:17:04 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 08 Feb 2015 12:17:04 +0200 Subject: Python is DOOMED! Again! References: <54c07d04$0$13012$c3e8da3$5496439d@news.astraweb.com> <54d6b1a0$0$6917$e4fe514c@dreader36.news.xs4all.nl> Message-ID: <87a90o6733.fsf@elektro.pacujo.net> Ian Kelly : > On Sat, Feb 7, 2015 at 5:45 PM, Albert van der Horst > wrote: >> x -> x**2 >> instead of >> lambda x : x**2 > > Well, I don't think the existing syntax is incompatible with your > proposal. As it is, the -> token can only appear after the argument > list of a def statement, so there would be no grammatical ambiguity. I > do think that such a proposal is unlikely to gain wide support though. I also don't think Python is in the need of any enhancement in this area. >> not >> def square(x): x**2 >> but >> square = x->x**2 > > This would be an anti-pattern. The def statement associates the name > "square" with the function's __name__ attribute, which is useful for > debugging and introspection. The proposed assignment statement does > not. A good point. However, Guile (Scheme) has some builtin magic: > (define (f x) x) > (procedure-name f) $1 = f > (define g (lambda (x) x)) > (procedure-name g) $2 = g > (define h g) > (procedure-name h) $3 = g >> mult = x,y -> >> result = 0 >> for i in range(x): >> result +=y >> return result > > I don't like this at all. The main problem is the missing colon, a cornerstone of Python syntax. You could have: mult = x,y ->: result = 0 for i in range(x): result +=y return result > I read "x -> x**2" as denoting a mapping from a bound variable to an > expression. A whole function body just feels wrong here. I don't think syntax like this is in any way wrong. It's just completely unnecessary given that we have "def". Some people are trying to make Scheme more Python-like, others are trying to make Python more Scheme-like. I think you should not dilute the idiomatic core of a programming language. When in Python, program in Python, when in Scheme, program in Scheme... Marko From flebber.crue at gmail.com Sun Feb 8 06:22:59 2015 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sun, 8 Feb 2015 03:22:59 -0800 (PST) Subject: lxml objectify - attribute elements to list. Message-ID: <69e3e0c1-5c40-43f1-9a7c-5efa80a481f8@googlegroups.com> Hi How can I actually access the values of an element with lxml objectify? for example if I had this element in my xml file. I can see all the attributes using this. In [86]: for child in root.getchildren(): print(child.attrib) ....: {} {'RequestCode': '', 'RequestId': '0'} {} {} ... {} {} {} {'Category': 'Metro', 'AbrClubDesc': 'VRC', 'State': 'VIC', 'ClubCode': '10018', 'Title': 'Victoria Racing Club'} {'TrackName': 'Main', 'VenueName': 'Flemington', 'TrackCode': '149', 'VenueAbbr': 'FLEM', 'VenueDesc': 'Flemington', 'VenueCode': '151'} {} {} ... Trying to access by attribs isn't working or me. In [90]: names = [p.text for p in root.Track.attrib['VenueName']] --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in () ----> 1 names = [p.text for p in root.Track.attrib['VenueName']] AttributeError: 'str' object has no attribute 'text' What am I missing with this? Thanks Sayth From kevin.p.dwyer at gmail.com Sun Feb 8 07:46:08 2015 From: kevin.p.dwyer at gmail.com (Kev Dwyer) Date: Sun, 08 Feb 2015 12:46:08 +0000 Subject: lxml objectify - attribute elements to list. References: <69e3e0c1-5c40-43f1-9a7c-5efa80a481f8@googlegroups.com> Message-ID: Sayth Renshaw wrote: > > Hi > > How can I actually access the values of an element with lxml objectify? > > for example if I had this element in my xml file. > > VenueCode="151" TrackName="Main" TrackCode="149"> > > I can see all the attributes using this. > > In [86]: for child in root.getchildren(): > print(child.attrib) > ....: > {} > {'RequestCode': '', 'RequestId': '0'} > {} > {} > ... > {} > {} > {} > {'Category': 'Metro', 'AbrClubDesc': 'VRC', 'State': 'VIC', 'ClubCode': > {'10018', 'Title': 'Victoria Racing Club'} 'TrackName': 'Main', > {'VenueName': 'Flemington', 'TrackCode': '149', 'VenueAbbr': 'FLEM', > {'VenueDesc': 'Flemington', 'VenueCode': '151'} } } > ... > > Trying to access by attribs isn't working or me. > > In [90]: names = [p.text for p in root.Track.attrib['VenueName']] > --------------------------------------------------------------------------- > AttributeError Traceback (most recent call > last) in () > ----> 1 names = [p.text for p in root.Track.attrib['VenueName']] > > AttributeError: 'str' object has no attribute 'text' > > > What am I missing with this? > > Thanks > > Sayth Hello Is this what you're trying to do? (tmp-179b92275909243d)kev at pluto ~/virtual-envs/tmp-179b92275909243d python Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import objectify >>> root = objectify.fromstring('') >>> root.Track '' >>> root >>> for child in root.getchildren(): ... print(child.attrib) ... {'VenueCode': '151', 'TrackCode': '149', 'VenueName': 'Flemington', 'VenueDesc': 'Flemington', 'TrackName': 'Main', 'VenueAbbr': 'FLEM'} >>> for child in root.getchildren(): ... print(child.get('VenueName')) ... Flemington >>> Cheers, Kev From fabien.maussion at gmail.com Sun Feb 8 08:41:48 2015 From: fabien.maussion at gmail.com (Fabien) Date: Sun, 08 Feb 2015 14:41:48 +0100 Subject: 'pip install nose' fails on python3 virtualenv Message-ID: Folks, I couldn't find any info on this problem online: Linux Mint Python 3.3 virtualenv (12.0.7) pip (6.0.8) setuptools (12.0.5) (testpy3)mowglie at flappi ~ $ pip install nose Collecting nose Exception: Traceback (most recent call last): File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/basecommand.py", line 232, in main status = self.run(options, args) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/commands/install.py", line 339, in run requirement_set.prepare_files(finder) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/req/req_set.py", line 333, in prepare_files upgrade=self.upgrade, File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/index.py", line 305, in find_requirement page = self._get_page(main_index_url, req) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/index.py", line 783, in _get_page return HTMLPage.get_page(link, req, session=self.session) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/index.py", line 872, in get_page "Cache-Control": "max-age=600", File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/requests/sessions.py", line 473, in get return self.request('GET', url, **kwargs) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/download.py", line 365, in request return super(PipSession, self).request(method, url, *args, **kwargs) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/requests/sessions.py", line 461, in request resp = self.send(prep, **send_kwargs) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/requests/sessions.py", line 573, in send r = adapter.send(request, **kwargs) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/cachecontrol/adapter.py", line 36, in send cached_response = self.controller.cached_request(request) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/cachecontrol/controller.py", line 100, in cached_request resp = self.serializer.loads(request, self.cache.get(cache_url)) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/cachecontrol/serialize.py", line 108, in loads return getattr(self, "_loads_v{0}".format(ver))(request, data) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/cachecontrol/serialize.py", line 148, in _loads_v1 return self.prepare_response(request, cached) File "/home/mowglie/.pyvirtualenvs/testpy3/lib/python3.3/site-packages/pip/_vendor/cachecontrol/serialize.py", line 129, in prepare_response body = io.BytesIO(cached["response"].pop("body")) TypeError: 'str' does not support the buffer interface I'm quite sure this problem appeared recently, I did not make any change to my system recently... Thanks, Fabien From dieffedue at gmx.com Sun Feb 8 09:06:05 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. EX SYZ GENEVE.) Date: Sun, 8 Feb 2015 06:06:05 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From yacinechaouche at yahoo.com Sun Feb 8 09:13:31 2015 From: yacinechaouche at yahoo.com (Yassine Chaouche) Date: Sun, 8 Feb 2015 06:13:31 -0800 (PST) Subject: How to write a non blocking SimpleHTTPRequestHandler ? In-Reply-To: References: <47031e69-e94d-4257-8c7d-e7c00a543634@googlegroups.com> <874mr3ibve.fsf@elektro.pacujo.net> <87r3u7grd3.fsf@elektro.pacujo.net> Message-ID: On Tuesday, February 3, 2015 at 3:17:37 PM UTC+1, Amirouche Boubekki wrote: > What you want is to prevent the socket to wait indefinetly for data (based on strace output) which is done with socket.setblocking/settimeout [1]. asynchronous (asyncio) is something else, and you would still need to handle blocking I think. I have installed Faulthandler, a beautiful tool written by Victor "Haypo" Stinner, and thanks to it I could determine precisely where the program hangs. It is in ssl.py::SSLSocket::read def read(self, len=1024): """Read up to LEN bytes and return them. Return zero-length string on EOF.""" try: > return self._sslobj.read(len) >*< python hangs on this line except SSLError, x: if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs: return '' else: raise >From the traceback given by faulthandler it seems to me that the problem isn't from my webserver trying to receive connections from clients, but from my server (acting as a client) trying a request on a distant server (https URL -> use of ssl.py). Here's the traceback : 1 Current thread 0x00007fb9cb41f700 (most recent call first): 2 File "/usr/lib/python2.7/ssl.py", line 160 in read 3 File "/usr/lib/python2.7/ssl.py", line 241 in recv 4 File "/usr/lib/python2.7/socket.py", line 447 in readline 5 File "/usr/lib/python2.7/httplib.py", line 365 in _read_status 6 File "/usr/lib/python2.7/httplib.py", line 407 in begin 7 File "/usr/lib/python2.7/httplib.py", line 1034 in getresponse 8 File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 353 in _make_request 9 File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 518 in urlopen 10 File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 370 in send 11 File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573 in send 12 File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 461 in request 13 File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 473 in get 14 File "/usr/local/lib/python2.7/dist-packages/infomaniak/infomaniak.py", line 29 in getFlux 15 File "/usr/local/lib/python2.7/dist-packages/infomaniak/server.py", line 52 in do_GET 16 File "/usr/lib/python2.7/BaseHTTPServer.py", line 328 in handle_one_request 17 File "/usr/lib/python2.7/BaseHTTPServer.py", line 340 in handle 18 File "/usr/lib/python2.7/SocketServer.py", line 649 in __init__ 19 File "/usr/lib/python2.7/SocketServer.py", line 334 in finish_request 20 File "/usr/lib/python2.7/SocketServer.py", line 321 in process_request 21 File "/usr/lib/python2.7/SocketServer.py", line 295 in _handle_request_noblock 22 File "/usr/lib/python2.7/SocketServer.py", line 238 in serve_forever 23 File "/usr/local/lib/python2.7/dist-packages/infomaniak/server.py", line 71 in 24 File "/usr/bin/infomaniak", line 2 in 25 If you look at line 13, it shows that the server is actually doing a get to an external URL via the requests library, which is itself relying on urllib3, which in turn is using httplib.py Below is the code of the last three functions to have been called, in chronoligical order : In socket.py::_fileobject::readline [...] while True: try: data = self._sock.recv(self._rbufsize) #<------------ except error, e: if e.args[0] == EINTR: continue raise if not data: break nl = data.find('\n') if nl >= 0: nl += 1 buf.write(data[:nl]) self._rbuf.write(data[nl:]) del data break buf.write(data) return buf.getvalue() [...] In ssl.py::SSLSocket::recv def recv(self, buflen=1024, flags=0): if self._sslobj: if flags != 0: raise ValueError( "non-zero flags not allowed in calls to recv() on %s" % self.__class__) return self.read(buflen) #<------- else: return self._sock.recv(buflen, flags) In ssl.py::SSLSocket::read def read(self, len=1024): """Read up to LEN bytes and return them. Return zero-length string on EOF.""" try: return self._sslobj.read(len) # >*< python hangs on this line except SSLError, x: if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs: return '' else: raise I can't go any further because the _sslobj is create via the _ssl.so library, it is very likely C code. Do you have any idea about how I can investigate this any further ? From dhamaniasad at gmail.com Sun Feb 8 14:17:00 2015 From: dhamaniasad at gmail.com (Asad Dhamani) Date: Sun, 8 Feb 2015 11:17:00 -0800 (PST) Subject: Profiler for long-running application Message-ID: <40a60076-8cc1-40a6-a0d4-29480b888039@googlegroups.com> I have a Flask application where I run a specific task asynchronously using Celery. Its basically parsing some HTML and inserting data into a Postgres database(using SQLAlchemy). However, the task seems to be running very slowly, at 1 insert per second. I'd like to find out where the bottleneck is, and I've been looking for a good profiler that'd let me do this, however, I couldn't find anything. Any recommendations would be great. From stefan_ml at behnel.de Sun Feb 8 14:29:42 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 08 Feb 2015 20:29:42 +0100 Subject: lxml objectify - attribute elements to list. In-Reply-To: <69e3e0c1-5c40-43f1-9a7c-5efa80a481f8@googlegroups.com> References: <69e3e0c1-5c40-43f1-9a7c-5efa80a481f8@googlegroups.com> Message-ID: Sayth Renshaw schrieb am 08.02.2015 um 12:22: > How can I actually access the values of an element with lxml objectify? > > for example if I had this element in my xml file. > > > > I can see all the attributes using this. > > In [86]: for child in root.getchildren(): > print(child.attrib) > ....: > {} > {'RequestCode': '', 'RequestId': '0'} > {} > {} > ... > {} > {} > {} > {'Category': 'Metro', 'AbrClubDesc': 'VRC', 'State': 'VIC', 'ClubCode': '10018', 'Title': 'Victoria Racing Club'} > {'TrackName': 'Main', 'VenueName': 'Flemington', 'TrackCode': '149', 'VenueAbbr': 'FLEM', 'VenueDesc': 'Flemington', 'VenueCode': '151'} > {} > {} > ... > > Trying to access by attribs isn't working or me. > > In [90]: names = [p.text for p in root.Track.attrib['VenueName']] > --------------------------------------------------------------------------- > AttributeError Traceback (most recent call last) > in () > ----> 1 names = [p.text for p in root.Track.attrib['VenueName']] > > AttributeError: 'str' object has no attribute 'text' As you can see from the output above, "attrib" is a mapping from strings (attribute names) to strings (attribute values). So just use name = root.Track.attrib['VenueName'] or, even simpler: name = root.Track.get('VenueName') Stefan From flebber.crue at gmail.com Sun Feb 8 15:49:33 2015 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sun, 8 Feb 2015 12:49:33 -0800 (PST) Subject: lxml objectify - attribute elements to list. In-Reply-To: References: <69e3e0c1-5c40-43f1-9a7c-5efa80a481f8@googlegroups.com> Message-ID: <215fd7b1-c71f-44a4-a576-05c4f226fb4a@googlegroups.com> Awesome, thanks so much for the help. Sayth From larry at hastings.org Sun Feb 8 17:00:33 2015 From: larry at hastings.org (Larry Hastings) Date: Sun, 08 Feb 2015 14:00:33 -0800 Subject: [RELEASE] Python 3.4.3rc1 is now available Message-ID: <54D7DC81.4000105@hastings.org> On behalf of the Python development community and the Python 3.4 release team, I'm happy to announce the availability of Python 3.4.3rc1. Python 3.4.3rc1 has many bugfixes and other small improvements over 3.4.2. You can download it here: https://www.python.org/download/releases/3.4.3 Not done yet, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From larry at hastings.org Sun Feb 8 17:03:05 2015 From: larry at hastings.org (Larry Hastings) Date: Sun, 08 Feb 2015 14:03:05 -0800 Subject: [RELEASE] Python 3.5.0a1 is now available Message-ID: <54D7DD19.5000400@hastings.org> On behalf of the Python development community and the Python 3.5 release team, I'm also pleased to announce the availability of Python 3.5.0a1. Python 3.5.0a1 is the first alpha release of Python 3.5, which will be the next major release of Python. Python 3.5 is still under heavy development, and is far from complete. This is a preview release, and its use is not recommended for production settings. You can download it here: https://www.python.org/download/releases/3.5.0a1 Happy hacking, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Sun Feb 8 17:06:36 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 08 Feb 2015 22:06:36 +0000 Subject: [RELEASE] Python 3.4.3rc1 is now available In-Reply-To: <54D7DC81.4000105@hastings.org> References: <54D7DC81.4000105@hastings.org> Message-ID: On 08/02/2015 22:00, Larry Hastings wrote: > > > On behalf of the Python development community and the Python 3.4 release > team, I'm happy to announce the availability of Python 3.4.3rc1. > Python 3.4.3rc1 has many bugfixes and other small improvements over 3.4.2. > > You can download it here: > > https://www.python.org/download/releases/3.4.3 > > > Not done yet, > > > //arry/ > The link provided took me to 3.4.1 not 3.4.3rc1. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From larry at hastings.org Sun Feb 8 17:14:52 2015 From: larry at hastings.org (Larry Hastings) Date: Sun, 08 Feb 2015 14:14:52 -0800 Subject: [RELEASE] Python 3.4.3rc1 is now available In-Reply-To: References: <54D7DC81.4000105@hastings.org> Message-ID: <54D7DFDC.8080104@hastings.org> On 02/08/2015 02:06 PM, Mark Lawrence wrote: > On 08/02/2015 22:00, Larry Hastings wrote: >> >> >> On behalf of the Python development community and the Python 3.4 release >> team, I'm happy to announce the availability of Python 3.4.3rc1. >> Python 3.4.3rc1 has many bugfixes and other small improvements over >> 3.4.2. >> >> You can download it here: >> >> https://www.python.org/download/releases/3.4.3 >> >> >> Not done yet, >> >> >> //arry/ >> > > The link provided took me to 3.4.1 not 3.4.3rc1. > Yeah, I made some sort of weird copy-and-paste error with the link. But if you type in https://www.python.org/download/releases/3.4.3 into your browser it works fine. Sorry for the inconvenience, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.stuart.85 at gmail.com Sun Feb 8 17:34:23 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Sun, 08 Feb 2015 22:34:23 +0000 Subject: Profiler for long-running application References: <40a60076-8cc1-40a6-a0d4-29480b888039@googlegroups.com> Message-ID: Hi Asad, Is there any reason why you can't just use profile/cProfile? In particular, you could use the api of that module to save out the profile stats to an external file with a unique name and then inspect them later with a tool like snakeviz . The code to save profile stats might look like the following: pr = cProfile.Profile() pr.runcall(your_celery_task_without_async) ps = pstats.Stats(pr) ps.dump_stats("my_task.profile") Obviously you need to call your celery task function directly, not via Celery using delay() or any derivative. Alternatively, you could try something like line_profiler by again, calling the task directly. If using this method it turn out your actual task code is running quite fast, then I'd suggest that the majority of the time is being lost in transferring the task to the Celery node. Cheers On Mon Feb 09 2015 at 5:20:43 AM Asad Dhamani wrote: > I have a Flask application where I run a specific task asynchronously > using Celery. Its basically parsing some HTML and inserting data into a > Postgres database(using SQLAlchemy). However, the task seems to be running > very slowly, at 1 insert per second. > I'd like to find out where the bottleneck is, and I've been looking for a > good profiler that'd let me do this, however, I couldn't find anything. Any > recommendations would be great. > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Sun Feb 8 18:08:56 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 08 Feb 2015 23:08:56 +0000 Subject: [RELEASE] Python 3.4.3rc1 is now available In-Reply-To: <54D7DFDC.8080104@hastings.org> References: <54D7DC81.4000105@hastings.org> <54D7DFDC.8080104@hastings.org> Message-ID: On 08/02/2015 22:14, Larry Hastings wrote: > On 02/08/2015 02:06 PM, Mark Lawrence wrote: >> On 08/02/2015 22:00, Larry Hastings wrote: >>> >>> On behalf of the Python development community and the Python 3.4 release >>> team, I'm happy to announce the availability of Python 3.4.3rc1. >>> Python 3.4.3rc1 has many bugfixes and other small improvements over >>> 3.4.2. >>> >>> You can download it here: >>> >>> https://www.python.org/download/releases/3.4.3 >>> >>> Not done yet, >>> >>> //arry/ >>> >> >> The link provided took me to 3.4.1 not 3.4.3rc1. >> > > Yeah, I made some sort of weird copy-and-paste error with the link. But > if you type in > > https://www.python.org/download/releases/3.4.3 > > into your browser it works fine. > > Sorry for the inconvenience, > > //arry/ > Damn and blast it, sorry I meant to say this earlier. If at the 3.4.3 page under the "Download" header you follow the link at "Please proceed to the download page for the download." you end up at https://www.python.org/downloads/release/python-33/ and "Error 404: File not Found" and "We couldn?t find what you were looking for. This error has been reported and we will look into it shortly." -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From skip.montanaro at gmail.com Sun Feb 8 22:44:57 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 8 Feb 2015 21:44:57 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db Message-ID: I am trying to process a CSV file using Python 3.5 (CPython tip as of a week or so ago). According to chardet[1], the file is encoded as utf-8: >>> s = open("data/meets-usms.csv", "rb").read() >>> len(s) 562272 >>> import chardet >>> chardet.detect(s) {'encoding': 'utf-8', 'confidence': 0.99} so I created the reader like so: rdr = csv.DictReader(open(csvfile, encoding="utf-8")) This seems to work. The rows are read and records added to a SQLite3 database. When I go into sqlite3, I get what looks to be raw utf-8 on output: % LANG=en_US.UTF-8 sqlite3 topten.db SQLite version 3.8.5 2014-08-15 22:37:57 Enter ".help" for usage hints. sqlite> select * from swimmeet where meetname like '%Barracuda%'; sqlite> select count(*) from swimmeet; 0 sqlite> select count(*) from swimmeet; 4171 sqlite> select meetname from swimmeet where meetname like '%Barracuda%Patrick%'; Anderson Barracudas St. Patrick's Day Swim Meet Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet Anderson Barracuda Masters 2011 St. Patrick???s Day Swim Meet Anderson Barracuda Masters St. Patrick's Day Meet Anderson Barracuda Masters St. Patrick's Day Meet 2014 Anderson Barracuda Masters 2015 St. Patrick???s Day Swim Meet Note the wacky three bytes where the apostrophe in "St. Patrick's" should be. The data came to me as an XLSX spreadsheet, which I dumped to CSV using LibreOffice. That's how the character was encoded at that point. I tweaked my CSV-to-SQLite script to print the meet name and id for those meets with "Barracuda" and "Patrick" in their name: if dry_run or verbose: if ("Barracuda" in row["MeetTitle"] and "Patrick" in row["MeetTitle"]): print("Insert", n, row["MeetTitle"], row["MeetID"]) When I run it, I see raw bytes instead of a properly rendered apostrophe: % LANG=en_US.utf-8 python3.5 src/usmsmeets2db.py -v data/meets-usms.csv topten.db Insert 1173 Anderson Barracudas St. Patrick's Day Swim Meet 20090321ABMSTPY Insert 1559 Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet 20100320CUDASY Insert 1995 Anderson Barracuda Masters 2011 St. Patrick???s Day Swim Meet 20110319ANDERSY Insert 3012 Anderson Barracuda Masters St. Patrick's Day Meet 20130316AndersY Insert 3562 Anderson Barracuda Masters St. Patrick's Day Meet 2014 20140315ANDERSY Insert 4114 Anderson Barracuda Masters 2015 St. Patrick???s Day Swim Meet 20150321AndersY Read 4962 rows, inserted 4171 records Why am I not seeing what I believe to be a non-ASCII apostrophe of some sort properly printed? This is running on a Mac (Yosemite) in its Terminal app, with its encoding preference set to utf-8. It appears just as shown above, "a" with a caret, the Euro symbol, then the "TM" symbol. Have I perhaps lost the properly encoded bytes somewhere, and now it's just spewing the bogus bytes (mojibake)? Thanks, Skip -- [1] https://pypi.python.org/pypi/chardet/2.3.0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sun Feb 8 22:58:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Feb 2015 14:58:44 +1100 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: Message-ID: On Mon, Feb 9, 2015 at 2:44 PM, Skip Montanaro wrote: > Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet Those three characters are the CP-1252 decode of the bytes for U+2019 in UTF-8 (E2 80 99). Not sure if that helps any, but given that it was an XLSX file, Windows codepages are reasonably likely to show up. ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 8 23:51:55 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 09 Feb 2015 15:51:55 +1100 Subject: Python 3.x stuffing utf-8 into SQLite db References: Message-ID: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Skip Montanaro wrote: > sqlite> select meetname from swimmeet where meetname like > '%Barracuda%Patrick%'; > Anderson Barracudas St. Patrick's Day Swim Meet > Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet > Anderson Barracuda Masters 2011 St. Patrick???s Day Swim Meet > Anderson Barracuda Masters St. Patrick's Day Meet > Anderson Barracuda Masters St. Patrick's Day Meet 2014 > Anderson Barracuda Masters 2015 St. Patrick???s Day Swim Meet > > Note the wacky three bytes where the apostrophe in "St. Patrick's" should > be. The data came to me as an XLSX spreadsheet, which I dumped to CSV > using LibreOffice. That's how the character was encoded at that point. The first question I would ask is whether SQLite's command shell supports UTF-8. I think it does, but aren't sure. The second question is, are you using Windows? If so, the Windows shell may be broken and mis-printing the string, or otherwise doing something weird: http://stackoverflow.com/questions/13405071/sqlite-converts-all-unicode- characters-into-ansi Can you confirm that the strings appear correctly in Python before you put them into sqlite? -- Steve From zhutieguo at gmail.com Sun Feb 8 23:52:50 2015 From: zhutieguo at gmail.com (zhutieguo at gmail.com) Date: Sun, 8 Feb 2015 20:52:50 -0800 (PST) Subject: I am a beginner, I designed a game is very interesting!! Message-ID: <1bfe36be-fac1-434b-bbbf-fce3d6cb34e0@googlegroups.com> I just started to learn python, and I designed a very interesting game "jail adventure". Give me some advise. Because English is not my first language, so you may find some grammar error in the story of the game. Here is the link http://pan.baidu.com/s/1i3gR6RF . press the button ???4KB) to download. From dieter at handshake.de Mon Feb 9 02:03:08 2015 From: dieter at handshake.de (dieter) Date: Mon, 09 Feb 2015 08:03:08 +0100 Subject: Profiler for long-running application References: <40a60076-8cc1-40a6-a0d4-29480b888039@googlegroups.com> Message-ID: <87fvafefdf.fsf@handshake.de> Asad Dhamani writes: > I'd like to find out where the bottleneck is, and I've been looking for a good profiler that'd let me do this, however, I couldn't find anything. Any recommendations would be great. Python comes with a "profile" module in its standard library. Implemented in Python and with quite a high overhead, it drastically slows down the execution, however. There is a C-implemented alternative (maybe named "cProfile", or something similar) - see the documentation of the "profile" module; when I remember right, it mentions alternatives. From tjreedy at udel.edu Mon Feb 9 02:38:29 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 09 Feb 2015 02:38:29 -0500 Subject: [RELEASE] Python 3.4.3rc1 is now available In-Reply-To: References: <54D7DC81.4000105@hastings.org> Message-ID: On 2/8/2015 5:06 PM, Mark Lawrence wrote: > On 08/02/2015 22:00, Larry Hastings wrote: >> >> >> On behalf of the Python development community and the Python 3.4 release >> team, I'm happy to announce the availability of Python 3.4.3rc1. >> Python 3.4.3rc1 has many bugfixes and other small improvements over >> 3.4.2. >> >> You can download it here: >> >> https://www.python.org/download/releases/3.4.3 >> >> >> Not done yet, >> >> >> //arry/ >> > > The link provided took me to 3.4.1 not 3.4.3rc1. This is the actual downloads page https://www.python.org/downloads/release/python-343rc1/ -- Terry Jan Reedy From dhamaniasad at gmail.com Mon Feb 9 04:13:07 2015 From: dhamaniasad at gmail.com (Asad Dhamani) Date: Mon, 9 Feb 2015 01:13:07 -0800 (PST) Subject: Profiler for long-running application In-Reply-To: References: <40a60076-8cc1-40a6-a0d4-29480b888039@googlegroups.com> Message-ID: <4cccb1d5-bcc6-4ad6-a4ca-86d684d68c74@googlegroups.com> On Monday, February 9, 2015 at 1:50:58 PM UTC+5:30, Ryan Stuart wrote: > Hi Asad, > > Is there any reason why you can't just use profile/cProfile? In particular, you could use the api of that module to save out the profile stats to an external file with a unique name and then inspect them later with a tool like snakeviz. The code to save profile stats might look like the following: > > > ? ? pr = cProfile.Profile() > ? ? pr.runcall(your_celery_task_without_async) > ? ? ps = pstats.Stats(pr) > ? ? ps.dump_stats("my_task.profile") > Obviously you need to call your celery task function directly, not via Celery using delay() or any derivative. Alternatively, you could try something like line_profiler?by again, calling the task directly. > > > If using this method it turn out your actual task code is running quite fast, then I'd suggest that the majority of the time is being lost in transferring the task to the Celery node. > > Cheers > > > On Mon Feb 09 2015 at 5:20:43 AM Asad Dhamani wrote: > I have a Flask application where I run a specific task asynchronously using Celery. Its basically parsing some HTML and inserting data into a Postgres database(using SQLAlchemy). However, the task seems to be running very slowly, at 1 insert per second. > > I'd like to find out where the bottleneck is, and I've been looking for a good profiler that'd let me do this, however, I couldn't find anything. Any recommendations would be great. > > -- > > https://mail.python.org/mailman/listinfo/python-list Hi Ryan, I was looking for something that didn't make me modify my code. It seems like I'll just use cProfile with snakeviz. Thanks for the help From rustompmody at gmail.com Mon Feb 9 06:28:41 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 9 Feb 2015 03:28:41 -0800 (PST) Subject: doc buglets? Message-ID: Poking around in help() (python 3.4.2) I find * PACKAGES does not seem to have anything on packages * DYNAMICFEATURES seems to be some kind of footnote * SPECIALATTRIBUTES has 'bases' and 'subclasses'. It seems to me a more consistent naming for OOP would be in order. These are the OOP-metaphors I am familiar with - superclass ? subclass - base class ? derived class - parent class ? child class [Yeah if this is a bug its more than a docbug. So no, I am not suggesting changing the actual attribute names which would be a breaking change, just the stuff in help() ] From jeanmichel at sequans.com Mon Feb 9 08:56:35 2015 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 9 Feb 2015 14:56:35 +0100 (CET) Subject: How to Mock a mongodb In-Reply-To: <54D68D06.70807@poblenousensefils.net> Message-ID: <1790626067.13376699.1423490195800.JavaMail.root@sequans.com> ----- Original Message ----- > From: "Xavier Pegenaute" > To: python-list at python.org > Sent: Saturday, 7 February, 2015 11:09:10 PM > Subject: How to Mock a mongodb > > Dear, > > I am trying to mock the use of a mongo db and I am having some > trouble. > Appears that I am not able to return a desired value from > mongo.find().count(). I made a proof of concept to try to reduce > complexity of the real problem. > > You can find the code which is going to be tested in [1], and the > code > of the test case in [2]. > > Do you know exactly wat's wrong with it?, or may you point me to some > direction? > > Thanks, > Xavi You've mocked the pymongo.cursor but since you've mocked the entire mongo client, this mocked client will never return a pymongo.cursor, it will return only mock objects. What you can do is mock the entire chain call of you mocked client: from mock import patch from mmongo import MongoHelper class TestMongoHelper(object): @patch("mmongo.pymongo") def test_count_data(self, fake_mongo_client): mhelper = MongoHelper() # mock self.db[MongoHelper.db_name][MongoHelper.coll_name].find().count() fake_mongo_client.__getitem__.return_value.__getitem__.return_value.find.return_value.count.return_value = 5 JM -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From fomcl at yahoo.com Mon Feb 9 10:43:55 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 9 Feb 2015 15:43:55 +0000 (UTC) Subject: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)? Message-ID: <1130125071.481542.1423496635653.JavaMail.yahoo@mail.yahoo.com> Hi, In the locale module we have: * setlocale, the setter that also returns something * getlocale, the getter that returns the OS-specific locale tuple (supposedly!) * getdefaultlocale, the getter that always returns a unix locale tuple Why are the getlocale() results below sometimes windows-like, sometimes unix-like? It seems that I need to use setlocale(), with only the 'category' parameter, right? ActivePython 3.3.2.0 (ActiveState Software Inc.) based on Python 3.3.2 (default, Sep 16 2013, 23:11:39) [MSC v.1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.getlocale() (None, None) # because setocale has not been called yet # works as expected >>> locale.setlocale(locale.LC_ALL, "") 'Dutch_Netherlands.1252' >>> locale.getlocale() ('Dutch_Netherlands', '1252') # bug!>>> locale.setlocale(locale.LC_ALL, "german") 'German_Germany.1252' >>> locale.getlocale() ('de_DE', 'cp1252') # incorect, unix-like! >>> locale.setlocale(locale.LC_ALL, "German_Germany.1252") 'German_Germany.1252' >>> locale.getlocale() ('de_DE', 'cp1252') # incorect, unix-like! # bug! >>> locale.setlocale(locale.LC_ALL, "spanish") 'Spanish_Spain.1252' >>> locale.getlocale() ('es_ES', 'cp1252') # incorect, unix-like! # works as expected >>> locale.setlocale(locale.LC_ALL, "italian") 'Italian_Italy.1252' >>> locale.getlocale() ('Italian_Italy', '1252') # correct! # ... maybe more? Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From breamoreboy at yahoo.co.uk Mon Feb 9 10:55:15 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 09 Feb 2015 15:55:15 +0000 Subject: doc buglets? In-Reply-To: References: Message-ID: On 09/02/2015 11:28, Rustom Mody wrote: > Poking around in help() (python 3.4.2) I find > > * PACKAGES > does not seem to have anything on packages > * DYNAMICFEATURES > seems to be some kind of footnote > * SPECIALATTRIBUTES > has 'bases' and 'subclasses'. It seems to me a more consistent naming > for OOP would be in order. These are the OOP-metaphors I am familiar with > - superclass ? subclass > - base class ? derived class > - parent class ? child class > > [Yeah if this is a bug its more than a docbug. > So no, I am not suggesting changing the actual attribute names which would be a > breaking change, just the stuff in help() > ] > So raise an issue and attach a patch. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Mon Feb 9 11:02:35 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 09 Feb 2015 16:02:35 +0000 Subject: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)? In-Reply-To: <1130125071.481542.1423496635653.JavaMail.yahoo@mail.yahoo.com> References: <1130125071.481542.1423496635653.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 09/02/2015 15:43, Albert-Jan Roskam wrote: > Hi, > > In the locale module we have: > * setlocale, the setter that also returns something > * getlocale, the getter that returns the OS-specific locale tuple (supposedly!) > > * getdefaultlocale, the getter that always returns a unix locale tuple > > Why are the getlocale() results below sometimes windows-like, sometimes unix-like? > It seems that I need to use setlocale(), with only the 'category' parameter, right? > > > ActivePython 3.3.2.0 (ActiveState Software Inc.) based on > Python 3.3.2 (default, Sep 16 2013, 23:11:39) [MSC v.1600 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import locale >>>> locale.getlocale() > (None, None) # because setocale has not been called yet > > > # works as expected > >>>> locale.setlocale(locale.LC_ALL, "") > 'Dutch_Netherlands.1252' >>>> locale.getlocale() > ('Dutch_Netherlands', '1252') > > # bug!>>> locale.setlocale(locale.LC_ALL, "german") > 'German_Germany.1252' >>>> locale.getlocale() > ('de_DE', 'cp1252') # incorect, unix-like! > > >>>> locale.setlocale(locale.LC_ALL, "German_Germany.1252") > 'German_Germany.1252' >>>> locale.getlocale() > ('de_DE', 'cp1252') # incorect, unix-like! > > > # bug! > >>>> locale.setlocale(locale.LC_ALL, "spanish") > 'Spanish_Spain.1252' >>>> locale.getlocale() > ('es_ES', 'cp1252') # incorect, unix-like! > > > # works as expected >>>> locale.setlocale(locale.LC_ALL, "italian") > 'Italian_Italy.1252' >>>> locale.getlocale() > ('Italian_Italy', '1252') # correct! > > > # ... maybe more? > > > Regards, > > Albert-Jan > There have been loads of bug reports on the issue tracker about locales. I suggest that you take a look to see if there is an open issue about this problem. > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a > > fresh water system, and public health, what have the Romans ever done for us? > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Those flaming Romans didn't get the locales right did they? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From info at wingware.com Mon Feb 9 11:24:53 2015 From: info at wingware.com (Wingware) Date: Mon, 09 Feb 2015 11:24:53 -0500 Subject: ANN: Wing IDE 5.1.1 released Message-ID: <54D8DF55.3060802@wingware.com> Hi, Wingware has released version 5.1.1 of Wing IDE, our cross-platform integrated development environment for the Python programming language. Wing IDE features a professional code editor with vi, emacs, visual studio, and other key bindings, auto-completion, call tips, context-sensitive auto-editing, goto-definition, find uses, refactoring, a powerful debugger, version control, unit testing, search, project management, and many other features. This minor release includes the following improvements: Improve Django support when settings is a package and not a module Preference to reuse existing instances of Wing when not otherwise specified Added documentation on backing up and sharing settings Only convert indents if pasting at start of line Fix auto-completion with double-click on Windows Fix restarting a debug session started with a Named Entry Point Fix autocomplete after a blank line in Python Shell and Debug Probe Keep editor scroll position the same when Replace All Fix color of current item highlight in the auto-completer Apply configured run marker alpha to selection color in Call Stack Fix auto-spacing around % in argument lists About 12 other bug fixes; see http://wingware.com/pub/wingide/5.1.1/CHANGELOG.txt What's New in Wing 5.1: Wing IDE 5.1 adds multi-process and child process debugging, syntax highlighting in the shells, persistent time-stamped unit test results, auto-conversion of indents on paste, an XCode keyboard personality, support for Flask and Django 1.7, and many other minor features and improvements. For details see http://wingware.com/news/2015-02-26 Free trial: http://wingware.com/wingide/trial Downloads: http://wingware.com/downloads Feature list: http://wingware.com/wingide/features Sales: http://wingware.com/store/purchase Upgrades: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From skip.montanaro at gmail.com Mon Feb 9 12:30:41 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 9 Feb 2015 11:30:41 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: Message-ID: On Sun, Feb 8, 2015 at 9:58 PM, Chris Angelico wrote: > Those three characters are the CP-1252 decode of the bytes for U+2019 > in UTF-8 (E2 80 99). Not sure if that helps any, but given that it was > an XLSX file, Windows codepages are reasonably likely to show up. Thanks, Chris. Are you telling me I should have defined the input file encoding for my CSV file as CP-1252, or that something got hosed on the export from XLSX to CSV? Or something else? Skip From skip.montanaro at gmail.com Mon Feb 9 12:32:29 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 9 Feb 2015 11:32:29 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> References: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 8, 2015 at 10:51 PM, Steven D'Aprano wrote: > The second question is, are you > using Windows? No, I'm on a Mac (as, I think I indicated in my original note). All transformations occurred on a Mac. LibreOffice spit out a CSV file (with those three odd bytes). My script sucked in the CSV file and inserted data into my SQLite db. Skip From rosuav at gmail.com Mon Feb 9 12:40:14 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 04:40:14 +1100 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: Message-ID: On Tue, Feb 10, 2015 at 4:30 AM, Skip Montanaro wrote: > On Sun, Feb 8, 2015 at 9:58 PM, Chris Angelico wrote: >> Those three characters are the CP-1252 decode of the bytes for U+2019 >> in UTF-8 (E2 80 99). Not sure if that helps any, but given that it was >> an XLSX file, Windows codepages are reasonably likely to show up. > > Thanks, Chris. Are you telling me I should have defined the input file > encoding for my CSV file as CP-1252, or that something got hosed on > the export from XLSX to CSV? Or something else? > > Skip Well, I'm not entirely sure. If your input file is actually CP-1252 and you try to decode it as UTF-8, you'll almost certainly get an error (unless of course it's all ASCII, but you know it isn't in this case). Also, I'd say chardet will be correct. But it might be worth locating one of those apostrophes in the file and looking at the actual bytes representing it... because what you may have is a crazy double-encoded system. If you take a document with U+2019 in it and encode it UTF-8, then decode it as CP-1252, then re-encode as UTF-8, you could get that. (I think. Haven't actually checked.) If someone gave UTF-8 bytes to a program that doesn't know the difference between bytes and characters, and assumes CP-1252, then you might well get something like this. Hence, having a look at the exact bytes in the .CSV file may help. Easiest might be to pull it up in a hex viewer (I use 'hd' on my Debian systems), and grep for the critical line. Otherwise, use Python and try to pull out a line from the byte stream. Good luck. You may need it. ChrisA From rosuav at gmail.com Mon Feb 9 12:41:40 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 04:41:40 +1100 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 10, 2015 at 4:32 AM, Skip Montanaro wrote: > On Sun, Feb 8, 2015 at 10:51 PM, Steven D'Aprano > wrote: >> The second question is, are you >> using Windows? > > No, I'm on a Mac (as, I think I indicated in my original note). All > transformations occurred on a Mac. LibreOffice spit out a CSV file > (with those three odd bytes). My script sucked in the CSV file and > inserted data into my SQLite db. Did the file originally come from Windows, or was it always on a Mac? ChrisA From chimex60 at gmail.com Mon Feb 9 12:43:09 2015 From: chimex60 at gmail.com (chimex60 at gmail.com) Date: Mon, 09 Feb 2015 18:43:09 +0100 Subject: Please help. Message-ID: <20150209174309.5931153.14331.26@gmail.com> An HTML attachment was scrubbed... URL: From mmr15 at case.edu Mon Feb 9 12:54:00 2015 From: mmr15 at case.edu (Matthew Ruffalo) Date: Mon, 09 Feb 2015 12:54:00 -0500 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: Message-ID: <54D8F438.1060402@case.edu> On 02/09/2015 12:30 PM, Skip Montanaro wrote: > Thanks, Chris. Are you telling me I should have defined the input file > encoding for my CSV file as CP-1252, or that something got hosed on > the export from XLSX to CSV? Or something else? > > Skip Hi Skip- I think it's most likely that the encoding issues happened in the export from XLSX to CSV (unless the data is malformed in the original XLSX file, of course). The file you're reading *is* valid UTF-8, and you're reading it in text mode, so seeing '???' in certain lines implies that the files might contain '???'.encode('utf-8') == b'\xc3\xa2\xe2\x82\xac\xe2\x84\xa2'. You could verify this with a hex editor, or use something like the following: """ #!/usr/bin/env python3 from argparse import ArgumentParser def dump_lines_with_high_bytes(filename): with open(filename, 'rb') as f: for line in f: if any(byte >= 0x80 for byte in line): print(line) if __name__ == '__main__': p = ArgumentParser() p.add_argument('filename') args = p.parse_args() dump_lines_with_high_bytes(args.filename) """ I'm a bit surprised that LibreOffice would mangle the encoding like this -- the entire point of using a format like XLSX is to avoid encoding issues. I just created a blank spreadsheet with Excel 2013 on Windows 7, pasted a U+2019 RIGHT SINGLE QUOTATION MARK into the top-left cell, saved it to a .xlsx file, and opened it with LibreOffice Calc 4.2.7.2 on Linux. The quot character displayed correctly, and I was able to save it to valid UTF-8 CSV that I could read with Python 3.4 without any mojibake. I wonder whether the encoding problems happened with whatever data was used to create your .xlsx file -- as Chris mentioned, this would occur if UTF-8 bytes are incorrectly decoded as Windows cp1252. MMR... From john_ladasky at sbcglobal.net Mon Feb 9 13:08:44 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 9 Feb 2015 10:08:44 -0800 (PST) Subject: Please help. In-Reply-To: References: Message-ID: On Monday, February 9, 2015 at 9:44:16 AM UTC-8, chim... at gmail.com wrote: > Hello. Am trying to change the key words to my tribal language. Eg change English language: print() to igbo language: de(). I have been stuck for months I need a mentor or someone that can guide me and answer some of my questions when I get stuck. Thanks.. > I will really appreciate it if someone attends to me. In Python, functions are bound to names, exactly like other variables are. You can bind any new name you want to an existing function, like this: Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print("English") English >>> de = print >>> de("Igbo") Igbo >>> print("Both function names work") Both function names work >>> de("Both function names work") Both function names work Hope that helps you. From brian.from.fl at gmail.com Mon Feb 9 13:11:12 2015 From: brian.from.fl at gmail.com (Brian) Date: Mon, 9 Feb 2015 10:11:12 -0800 (PST) Subject: Python 3 and the requests library Message-ID: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> On the Mac running Mavericks, I have successfully managed to install and use the requests library for HTTP and HTTPS requests using Python 2. But I'd like to move to Python 3. I downloaded the most recent stable version of Python 3 for Mac. All is well. I did a pip3 install requests command and it worked. I could then start the interpreter and perform a simple HTTP GET request: $ python3 Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 5 2014, 20:42:22) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> r = requests.get("http://www.google.com") >>> r So, with that success, I created an executable script filled with the same commands: #!/usr/bin/env python3 import requests r = requests.get("http://www.google.com") print(r) At first it failed because it couldn't find httplib2. Not sure why, but I installed httplib2 and now get the following error: $ ./test.py Traceback (most recent call last): File "test.py", line 3, in import requests File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/__init__.py", line 53, in from .packages.urllib3.contrib import pyopenssl File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/__init__.py", line 3, in from . import urllib3 File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/__init__.py", line 10, in from .connectionpool import ( File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 2, in import logging File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/logging/__init__.py", line 26, in import sys, os, time, io, traceback, warnings, weakref, collections File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 3, in import linecache File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/linecache.py", line 10, in import tokenize File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py", line 40, in __all__ = token.__all__ + ["COMMENT", "tokenize", "detect_encoding", AttributeError: 'module' object has no attribute '__all__' Here is the full list of installed packages that I have in python3 (the list in python2 is huge and not posted here): $ pip3 list httplib2 (0.9) pip (1.5.6) requests (2.5.1) setuptools (2.1) urllib3 (1.10) Notre that if I change python3 to python in my script, it works fine using the default Python 2.7.5 version. And as a Plan B I can stay with Python 2.7.5. But I'd really like to move to Python 3 on the Mac. Thanks in advance for any thoughts or help! Brian From sohcahtoa82 at gmail.com Mon Feb 9 13:23:06 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Mon, 9 Feb 2015 10:23:06 -0800 (PST) Subject: How to extract a movie title out of a text file In-Reply-To: References: Message-ID: <68d7953f-55fc-40b0-aaf2-2bd8e46ff22c@googlegroups.com> On Saturday, February 7, 2015 at 8:54:26 AM UTC-8, Seymore4Head wrote: > What I would like to be able to do is download and save to a folder > all the srr files in this Usenet group. alt.binaries.moovee > > I would then like to be able to search for a title in the text file. > > Anyone care to come up with the Python code to do this? This guy's posts are always good for a laugh. I hadn't seen one in a while and I was starting to miss them. From d at davea.name Mon Feb 9 13:25:40 2015 From: d at davea.name (Dave Angel) Date: Mon, 09 Feb 2015 13:25:40 -0500 Subject: Please help. In-Reply-To: References: Message-ID: <54D8FBA4.6050708@davea.name> On 02/09/2015 01:08 PM, John Ladasky wrote: > On Monday, February 9, 2015 at 9:44:16 AM UTC-8, chim... at gmail.com wrote: >> Hello. Am trying to change the key words to my tribal language. Eg change English language: print() to igbo language: de(). I have been stuck for months I need a mentor or someone that can guide me and answer some of my questions when I get stuck. Thanks.. >> I will really appreciate it if someone attends to me. > > > In Python, functions are bound to names, exactly like other variables are. You can bind any new name you want to an existing function, like this: > > Python 3.4.0 (default, Apr 11 2014, 13:05:11) > [GCC 4.8.2] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> print("English") > English >>>> de = print >>>> de("Igbo") > Igbo >>>> print("Both function names work") > Both function names work >>>> de("Both function names work") > Both function names work > > > Hope that helps you. > That will help him with the functions. But not with the keywords. The OP didn't specify Python version, but in 3.x, print() is a function, and can be rebound. Since he said keyword, he's either mistaken, or he's running 2.x But any real keywords, are a real problem. For example, changing 'if' to some other string. And all the library code is a problem, as they have to be individually translated. And many of them are not fully in Python, but have C portions. Then there's the return strings for exceptions, and file names for imports. And the documentation, and the inline documentation, and reflection. Many problems, no good solutions. It has been considered many times by many good Python developers, and no reasonable solution has presented itself. When I use grep at the bash command line, do I want these translated to English words. Nope. They both are acronyms, but few people know what they actually stand for. I wish there were an answer, but I don't think so, other than using a non-language (like Esperanto) to pick all the keywords and library functions of a new language in. Then that new language would be equally hard for all nationalities to learn. http://legacy.python.org/workshops/1997-10/proceedings/loewis.html http://grokbase.com/t/python/python-list/09bsr7hjwh/python-statements-keyword-localization -- DaveA From skip.montanaro at gmail.com Mon Feb 9 13:52:54 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 9 Feb 2015 12:52:54 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: <54D8F438.1060402@case.edu> References: <54D8F438.1060402@case.edu> Message-ID: On Mon, Feb 9, 2015 at 11:54 AM, Matthew Ruffalo wrote: > I think it's most likely that the encoding issues happened in the export > from XLSX to CSV (unless the data is malformed in the original XLSX > file, of course). Aha! Lookee here... (my apologies to all you HTML mail haters - sometimes it even comes in handy. :-) This snapshot was taken against a running LibreOffice instance here at work (on Linux). It would appear the fancy schmancy apostrophe was hosed up before the data ever got to me. Had a guy here with Windows pop up the original file I got in an actual Excel instance. Same bogosity. Knowing that, I don't feel the least bit timid about just editing the darn CSV file to correct the encode/decode/encode error before loading the data into SQLite. I was worried that since I was doing some Python 3.x stuff involving Unicode for the first time that I'd screwed something up. Skip ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: stpat.png Type: image/png Size: 47543 bytes Desc: not available URL: From john_ladasky at sbcglobal.net Mon Feb 9 14:06:32 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 9 Feb 2015 11:06:32 -0800 (PST) Subject: Please help. In-Reply-To: References: Message-ID: <0f4bf4c1-c87e-46dc-bd92-00ab0d599a93@googlegroups.com> On Monday, February 9, 2015 at 10:26:47 AM UTC-8, Dave Angel wrote: > That will help him with the functions. But not with the keywords. The > OP didn't specify Python version, but in 3.x, print() is a function, and > can be rebound. Since he said keyword, he's either mistaken, or he's > running 2.x > > But any real keywords, are a real problem. For example, changing 'if' > to some other string. That's true, Dave, he did say "key words". I only helped him with his example of a Python built-in function. And you're right, shadowing key words is currently not possible. Are you familiar with a program known as a "talk filter"? They've been around for decades. http://www.hyperrealm.com/talkfilters/talkfilters.html Talk filters are usually used for off-color humor, rather than for serious purposes. A talk filter processes a text stream, line-by-line, typically in a chat room setting. A dictionary of words, and their replacements, is defined. I could see a talk filter being used to translate a non-English version of Python into standard English Python code. I suppose that a hook for a talk filter could be added to the interpreter itself. It wouldn't be easy to change word order to accommodate the syntax of another language, but words themselves could be altered. From bv8bv8bv8 at gmail.com Mon Feb 9 14:12:23 2015 From: bv8bv8bv8 at gmail.com (BV BV) Date: Mon, 9 Feb 2015 11:12:23 -0800 (PST) Subject: WHO IS PROPHET MUHAMMAD, MAY PEACE AND BLESSINGS BE UPON HIM? Message-ID: WHO IS PROPHET MUHAMMAD, MAY PEACE AND BLESSINGS BE UPON HIM? He is the one who defended the rights of all humanity 1400 years ago He defended men's, women's and children rights He commanded and fostered the love between relatives and neighbors He established a coexistence relationship between Muslims and Non-Muslims He organized the relationship between the members of the family putting duties on sons and daughters towards the parents He fought injustice, called for justice, love, unity and cooperation for the good. He called for helping the needy, visiting the patients, love and exchanging advises between people. He prohibited (by orders from God) bad manners such as stealing, lying, torturing and murdering. He is the one who changed our lives and manners to be better. A Muslim doesn't steal A Muslim doesn't lie A Muslim doesn't drink alcohol. A Muslim doesn't commit adultery A Muslim doesn't cheat A Muslim doesn't kill innocent people A Muslim doesn't harm his neighbors A Muslim obeys his parents and helps them A Muslim is kind to young and elderly people, to women and to weak people. A Muslim doesn't torture humans or even animals, and does not harm trees A Muslim loves his wife and takes care of his children and show mercy towards them until the last day of his life. A Muslim's relationship towards his children never stops even when they become adults He is Muhammad (PBUH) Did you know why all Muslims love Muhammad (PBUH)? Did you know what does Muhammad mean for Muslims? Every Muslim loves Muhammad (peace be upon him) more than himself and more than everything in his life. Before judging a Muslim be fair and: 1-Listen to this person, and watch his doings. 2-Compare his ideas and teachings with what is Islam and Prophet Mohammad PBUH ordered. 3-If you think that his thoughts are typical to that of Islam and Prophet Mohammad PBUH, and then compare them with his doings; is he applying these teachings? 4-If he is applying these teachings and sayings, so for sure represents Islam, if not then he calls himself a Muslim but doesn't represent Islam. Mohammed Ansan and far from extremism and lying which has marketed We as Muslims we publish a genuine people's love you oh Lord weiamhamd Than you From charleshixsn at earthlink.net Mon Feb 9 14:14:55 2015 From: charleshixsn at earthlink.net (Charles Hixson) Date: Mon, 09 Feb 2015 11:14:55 -0800 Subject: __next__ and StopIteration Message-ID: <54D9072F.3010904@earthlink.net> I'm trying to write a correct iteration over a doubly indexed container, and what I've got so far is: def __next__ (self): for row in range(self._rows): for col in range(self._cols): if self._grid[row][col]: yield self._grid[row][col] #end if #end for col #end for row raise StopIteration What bothers me is that it doesn't look like it would continue to raise StopIteration if it were called again, which is what https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says is correct. How should this be fixed? From brian.from.fl at gmail.com Mon Feb 9 14:20:54 2015 From: brian.from.fl at gmail.com (Brian) Date: Mon, 9 Feb 2015 11:20:54 -0800 (PST) Subject: Python 3 and the requests library In-Reply-To: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> Message-ID: <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> I am also seeing this in my Mac Mavericks Python 3 installation when I use just the built-in logging library. Again, a tiny example executable script and the results: $ cat test2.py #!/usr/bin/env python3 import logging logging.info("TEST2 starting") $ ./test2.py Traceback (most recent call last): File "./test2.py", line 3, in import logging File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/logging/__init__.py", line 26, in import sys, os, time, io, traceback, warnings, weakref, collections File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 3, in import linecache File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/linecache.py", line 10, in import tokenize File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py", line 40, in __all__ = token.__all__ + ["COMMENT", "tokenize", "detect_encoding", AttributeError: 'module' object has no attribute '__all__' Googling hasn't helped track this one down. In lieu of an answer, some pointers to tools or other things to look for would be greatly appreciated. Thanks! Brian From ned at nedbatchelder.com Mon Feb 9 14:24:04 2015 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 09 Feb 2015 14:24:04 -0500 Subject: __next__ and StopIteration In-Reply-To: <54D9072F.3010904@earthlink.net> References: <54D9072F.3010904@earthlink.net> Message-ID: On 2/9/15 2:14 PM, Charles Hixson wrote: > I'm trying to write a correct iteration over a doubly indexed container, > and what I've got so far is: def __next__ (self): > for row in range(self._rows): > for col in range(self._cols): > if self._grid[row][col]: > yield self._grid[row][col] > #end if > #end for col > #end for row > raise StopIteration > > What bothers me is that it doesn't look like it would continue to raise > StopIteration if it were called again, which is what > https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says > is correct. How should this be fixed? You are using yield, which means __next__ is a generator. As such, you don't have to explicitly raise StopIteration at all. Just remove that statement, and you should be fine. Also, look into how you are posting, the code is nearly mangled. :( -- Ned Batchelder, http://nedbatchelder.com From rgaddi at technologyhighland.invalid Mon Feb 9 14:27:04 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Mon, 9 Feb 2015 19:27:04 +0000 (UTC) Subject: __next__ and StopIteration References: Message-ID: On Mon, 09 Feb 2015 11:14:55 -0800, Charles Hixson wrote: > I'm trying to write a correct iteration over a doubly indexed container, > and what I've got so far is: def __next__ (self): > for row in range(self._rows): > for col in range(self._cols): > if self._grid[row][col]: > yield self._grid[row][col] > #end if > #end for col > #end for row raise StopIteration > > What bothers me is that it doesn't look like it would continue to raise > StopIteration if it were called again, which is what > https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says > is correct. How should this be fixed? You're mixing metaphors. You don't iterate over containers, you iterate over iterators that are returned to you by containers. So the container class itself has a __iter__ method, which returns a custom iterator object that has a __next__ method. Which is a lot of work. Or, you write your __iter__ like so: def __iter__ (self): for row in range(self._rows): for col in range(self._cols): if self._grid[row][col]: yield self._grid[row][col] In which case, Python's magic handling of yield handles the creation of the iterator object and the raising of StopIteration at the end of the __iter__ function all by itself. Or you write your __iter__ like so: def __iter__(self): return (self._grid[row][col] for col in range(self._cols) for row in range(self._rows) if self._grid[row][col] ) In which case you've returned a generator expression, which is a shorthand way of making an iterator. All extremely equivalent and a matter of personal taste (I'd probably opt for the yield one myself). -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From none at mailinator.com Mon Feb 9 14:41:20 2015 From: none at mailinator.com (mm0fmf) Date: Mon, 09 Feb 2015 19:41:20 +0000 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: Message-ID: On 09/02/2015 03:44, Skip Montanaro wrote: > I am trying to process a CSV file using Python 3.5 (CPython tip as of a > week or so ago). According to chardet[1], the file is encoded as utf-8: > > >>> s = open("data/meets-usms.csv", "rb").read() > >>> len(s) > 562272 > >>> import chardet > >>> chardet.detect(s) > {'encoding': 'utf-8', 'confidence': 0.99} > > so I created the reader like so: > > rdr = csv.DictReader(open(csvfile, encoding="utf-8")) > > This seems to work. The rows are read and records added to a SQLite3 > database. When I go into sqlite3, I get what looks to be raw utf-8 on > output: > > % LANG=en_US.UTF-8 sqlite3 topten.db > SQLite version 3.8.5 2014-08-15 22:37:57 > Enter ".help" for usage hints. > sqlite> select * from swimmeet where meetname like '%Barracuda%'; > sqlite> select count(*) from swimmeet; > 0 > sqlite> select count(*) from swimmeet; > 4171 > sqlite> select meetname from swimmeet where meetname like > '%Barracuda%Patrick%'; > Anderson Barracudas St. Patrick's Day Swim Meet > Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet > Anderson Barracuda Masters 2011 St. Patrick???s Day Swim Meet > Anderson Barracuda Masters St. Patrick's Day Meet > Anderson Barracuda Masters St. Patrick's Day Meet 2014 > Anderson Barracuda Masters 2015 St. Patrick???s Day Swim Meet > How is meetname defined? Is it a varchar or nvarchar? My only experience is with MS-SQL and C# but reading from a utf-8 encoded file with a StreamReader set to utf-8 and trying to insert that into varchar fields results in similar issues to what you are showing. I changed to using nvarchar and it all start working as expected. From zachary.ware+pylist at gmail.com Mon Feb 9 15:05:43 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 9 Feb 2015 14:05:43 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 9, 2015 at 11:32 AM, Skip Montanaro wrote: > LibreOffice spit out a CSV file > (with those three odd bytes). My script sucked in the CSV file and > inserted data into my SQLite db. If all else fails, you can try ftfy to fix things: http://ftfy.readthedocs.org/en/latest/ >>> import ftfy >>> ftfy.fix_text('Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet') "Anderson Barracuda Masters - 2010 St. Patrick's Day Swim Meet" It also seems to agree that there was a bad (en|de)coding with cp1252 at some point. >>> ftfy.fixes.fix_encoding_and_explain('Anderson Barracuda Masters - 2010 St. Patrick???s Day Swim Meet') ('Anderson Barracuda Masters - 2010 St. Patrick?s Day Swim Meet', [('encode', 'sloppy-windows-1252'), ('decode', 'utf-8')]) -- Zach From zachary.ware+pylist at gmail.com Mon Feb 9 15:11:40 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 9 Feb 2015 14:11:40 -0600 Subject: Python 3 and the requests library In-Reply-To: <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> Message-ID: On Mon, Feb 9, 2015 at 1:20 PM, Brian wrote: > I am also seeing this in my Mac Mavericks Python 3 installation when I use just the built-in logging library. Again, a tiny example executable script and the results: > > $ cat test2.py > #!/usr/bin/env python3 > import logging > logging.info("TEST2 starting") > > $ ./test2.py > Traceback (most recent call last): > File "./test2.py", line 3, in > import logging > File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/logging/__init__.py", line 26, in > import sys, os, time, io, traceback, warnings, weakref, collections > File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 3, in > import linecache > File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/linecache.py", line 10, in > import tokenize > File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/tokenize.py", line 40, in > __all__ = token.__all__ + ["COMMENT", "tokenize", "detect_encoding", > AttributeError: 'module' object has no attribute '__all__' > > Googling hasn't helped track this one down. In lieu of an answer, some pointers to tools or other things to look for would be greatly appreciated. Thanks! Try this, from wherever test2.py lives: python3 -c "import token;print(token.__file__)" You should get back something akin to "/usr/lib64/python3.4/token.py" (however that translates to Mac). If instead you get the path to some file of your own, rename your file. Hope this helps, -- Zach From fomcl at yahoo.com Mon Feb 9 15:35:49 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 9 Feb 2015 20:35:49 +0000 (UTC) Subject: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)? In-Reply-To: References: Message-ID: <336625397.524297.1423514149338.JavaMail.yahoo@mail.yahoo.com> ----- Original Message ----- > From: Mark Lawrence > To: python-list at python.org > Cc: > Sent: Monday, February 9, 2015 5:02 PM > Subject: Re: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)? > > On 09/02/2015 15:43, Albert-Jan Roskam wrote: >> Hi, >> >> In the locale module we have: >> * setlocale, the setter that also returns something >> * getlocale, the getter that returns the OS-specific locale tuple > (supposedly!) >> >> * getdefaultlocale, the getter that always returns a unix locale tuple >> >> Why are the getlocale() results below sometimes windows-like, sometimes > unix-like? >> It seems that I need to use setlocale(), with only the 'category' > parameter, right? >> >> >> ActivePython 3.3.2.0 (ActiveState Software Inc.) based on >> Python 3.3.2 (default, Sep 16 2013, 23:11:39) [MSC v.1600 64 bit (AMD64)] > on win32 >> Type "help", "copyright", "credits" or > "license" for more information. >>>>> import locale >>>>> locale.getlocale() >> (None, None) # because setocale has not been called yet >> >> >> # works as expected >> >>>>> locale.setlocale(locale.LC_ALL, "") >> 'Dutch_Netherlands.1252' >>>>> locale.getlocale() >> ('Dutch_Netherlands', '1252') >> >> # bug!>>> locale.setlocale(locale.LC_ALL, "german") >> 'German_Germany.1252' >>>>> locale.getlocale() >> ('de_DE', 'cp1252') # incorect, unix-like! >> >> >>>>> locale.setlocale(locale.LC_ALL, > "German_Germany.1252") >> 'German_Germany.1252' >>>>> locale.getlocale() >> ('de_DE', 'cp1252') # incorect, unix-like! >> >> >> # bug! >> >>>>> locale.setlocale(locale.LC_ALL, "spanish") >> 'Spanish_Spain.1252' >>>>> locale.getlocale() >> ('es_ES', 'cp1252') # incorect, unix-like! >> >> >> # works as expected >>>>> locale.setlocale(locale.LC_ALL, "italian") >> 'Italian_Italy.1252' >>>>> locale.getlocale() >> ('Italian_Italy', '1252') # correct! >> >> >> # ... maybe more? >> >> >> Regards, >> >> Albert-Jan >> > > There have been loads of bug reports on the issue tracker about locales. > I suggest that you take a look to see if there is an open issue about > this problem. Hi Mark -Thanks! I checked the bug tracker. Lots of locale-related bugs indeed. I couldn't find this one though. Strange because what I am doing does not seem exotic at all. French, German, Portuguese and Spanish getlocale results are incorrect IMHO. Below is little check + output here in case it might be useful for others. This is with Python 3.4, 3.3 and 2.7 on Windows 7 64. from __future__ import print_function import locale import collections import pprint #https://msdn.microsoft.com/en-us/library/39cwe7zf%28v=vs.80%29.aspx languages = ("chinese czech danish dutch english finnish french german greek " "hungarian icelandic italian japanese korean norwegian polish " "portuguese russian slovak spanish swedish turkish") d = collections.defaultdict(list) t = collections.namedtuple("Locale", "lang setlocale getlocale") for language in languages.split(): sloc = locale.setlocale(locale.LC_ALL, language) gloc = locale.getlocale() record = t(language, sloc, gloc) if gloc[0][2] == "_": d["unix-like"].append(record) else: d["windows-like"].append(record) pprint.pprint(dict(d)) n:\>C:\Miniconda3\python.exe N:\temp\loc.py {'unix-like': [Locale(lang='french', setlocale='French_France.1252', getlocale=('fr_FR', 'cp1252')), Locale(lang='german', setlocale='German_Germany.1252', getlocale=('de_DE', 'cp1252')), Locale(lang='portuguese', setlocale='Portuguese_Brazil.1252', getlocale=('pt_BR', 'cp1252')), Locale(lang='spanish', setlocale='Spanish_Spain.1252', getlocale=('es_ES', 'cp1252'))], 'windows-like': [Locale(lang='chinese', setlocale="Chinese (Simplified)_People's Republic of China.936", getlocale=("Chinese (Simplified)_People's Republic of China", '936')), Locale(lang='czech', setlocale='Czech_Czech Republic.1250', getlocale=('Czech_Czech Republic', '1250')), Locale(lang='danish', setlocale='Danish_Denmark.1252', getlocale=('Danish_Denmark', '1252')), Locale(lang='dutch', setlocale='Dutch_Netherlands.1252', getlocale=('Dutch_Netherlands', '1252')), Locale(lang='english', setlocale='English_United States.1252', getlocale=('English_United States', '1252')), Locale(lang='finnish', setlocale='Finnish_Finland.1252', getlocale=('Finnish_Finland', '1252')), Locale(lang='greek', setlocale='Greek_Greece.1253', getlocale=('Greek_Greece', '1253')), Locale(lang='hungarian', setlocale='Hungarian_Hungary.1250', getlocale=('Hungarian_Hungary', '1250')), Locale(lang='icelandic', setlocale='Icelandic_Iceland.1252', getlocale=('Icelandic_Iceland', '1252')), Locale(lang='italian', setlocale='Italian_Italy.1252', getlocale=('Italian_Italy', '1252')), Locale(lang='japanese', setlocale='Japanese_Japan.932', getlocale=('Japanese_Japan', '932')), Locale(lang='korean', setlocale='Korean_Korea.949', getlocale=('Korean_Korea', '949')), Locale(lang='norwegian', setlocale='Norwegian (Bokm?l)_Norway.1252', getlocale=('Norwegian (Bokm?l)_Norway', '1252')), Locale(lang='polish', setlocale='Polish_Poland.1250', getlocale=('Polish_Poland', '1250')), Locale(lang='russian', setlocale='Russian_Russia.1251', getlocale=('Russian_Russia', '1251')), Locale(lang='slovak', setlocale='Slovak_Slovakia.1250', getlocale=('Slovak_Slovakia', '1250')), Locale(lang='swedish', setlocale='Swedish_Sweden.1252', getlocale=('Swedish_Sweden', '1252')), Locale(lang='turkish', setlocale='Turkish_Turkey.1254', getlocale=('Turkish_Turkey', '1254'))]} From brian.from.fl at gmail.com Mon Feb 9 15:37:00 2015 From: brian.from.fl at gmail.com (Brian) Date: Mon, 9 Feb 2015 12:37:00 -0800 (PST) Subject: Python 3 and the requests library In-Reply-To: References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> Message-ID: <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> Zach, Here is what I get on the Mac: $ python3 -c "import token;print(token.__file__)" /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/token.py Just for grins, I also ran it against the built-in Python 2.7.5 version: $ python -c "import token;print(token.__file__)" /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/token.pyc The only difference seems to be that the 2.7.5 version has precompiled it. Then I followed your example but ran it against the logging module instead, as that seems to be where the problem lies. Again, for both version (broken 3 and working 2) as a comparison: $ python3 -c "import logging;print(logging.__file__)" /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/logging/__init__.py $ python -c "import logging;print(logging.__file__)" /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.pyc But nothing is pointing to anything except what appears to be the formally installed versions of these library modules. Brian From ian.g.kelly at gmail.com Mon Feb 9 15:58:04 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 9 Feb 2015 13:58:04 -0700 Subject: Python 3 and the requests library In-Reply-To: <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> Message-ID: On Mon, Feb 9, 2015 at 1:37 PM, Brian wrote: > Zach, > > Here is what I get on the Mac: > > $ python3 -c "import token;print(token.__file__)" > /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/token.py Are you running this from the same directory where you have your test scripts (in case there is also a token module hanging around in that directory)? From zachary.ware+pylist at gmail.com Mon Feb 9 16:03:33 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 9 Feb 2015 15:03:33 -0600 Subject: Python 3 and the requests library In-Reply-To: <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> Message-ID: On Mon, Feb 9, 2015 at 2:37 PM, Brian wrote: > Zach, > > Here is what I get on the Mac: > > $ python3 -c "import token;print(token.__file__)" > /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/token.py That looks correct. > Just for grins, I also ran it against the built-in Python 2.7.5 version: > > $ python -c "import token;print(token.__file__)" > /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/token.pyc > > The only difference seems to be that the 2.7.5 version has precompiled it. The 3.4 version is also precompiled, but __file__ gives the filename of the actual module (not the .pyc implementation detail). You can find precompiled token.py for 3.4 at /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/__pycache__/token.cpython-34.pyc > Then I followed your example but ran it against the logging module instead, as that > seems to be where the problem lies. Not so, the traceback you posted shows pretty clearly that the problem is in the tokenize module, where the token module it has imported does not have an __all__ attribute. > Again, for both version (broken 3 and working 2) as a comparison: > > $ python3 -c "import logging;print(logging.__file__)" > /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/logging/__init__.py This should have failed with the same traceback as your test2.py. > $ python -c "import logging;print(logging.__file__)" > /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.pyc > > But nothing is pointing to anything except what appears to be the formally installed > versions of these library modules. Try your test again. If it fails again, run my test again at the same prompt. -- Zach From ethan at stoneleaf.us Mon Feb 9 16:28:37 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 09 Feb 2015 13:28:37 -0800 Subject: What killed Smalltalk could kill Python In-Reply-To: References: <54bfd513$0$12978$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54D92685.3010301@stoneleaf.us> On 01/21/2015 03:37 PM, Tim Daneliuk wrote: > > I wrote some rambling disquisition on these matters some years ago ... > > http://www.tundraware.com/TechnicalNotes/Python-Is-Middleware > > http://www.tundraware.com/TechnicalNotes/How-To-Pick-A-Programming-Language Very enjoyable, thank you! -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From brian.from.fl at gmail.com Mon Feb 9 16:37:56 2015 From: brian.from.fl at gmail.com (Brian) Date: Mon, 9 Feb 2015 13:37:56 -0800 (PST) Subject: Python 3 and the requests library In-Reply-To: References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> Message-ID: Thank you, Ian and Zack! That was exactly the issue. Apparently, having a token.py script (one of my first Python 2 scripts to get an authorization token from a billing server) is OK for Python 2 but breaks Python 3. *face palm* Thank you again so very much! Brian On Monday, February 9, 2015 at 3:59:11 PM UTC-5, Ian wrote: > On Mon, Feb 9, 2015 at 1:37 PM, Brian wrote: > > Zach, > > > > Here is what I get on the Mac: > > > > $ python3 -c "import token;print(token.__file__)" > > /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/token.py > > Are you running this from the same directory where you have your test > scripts (in case there is also a token module hanging around in that > directory)? From p.f.moore at gmail.com Mon Feb 9 16:57:34 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Feb 2015 13:57:34 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: <54d55314$0$12979$c3e8da3$5496439d@news.astraweb.com> References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> <51d12f89-7d03-41e9-a2c9-3960a15ed6aa@googlegroups.com> <54d55314$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Friday, 6 February 2015 23:49:51 UTC, Steven D'Aprano wrote: > > Just a quick status update, in case you're interested. With relatively > > little work (considering I started not knowing much about numpy) I managed > > to put together solutions for a couple of my friend's problems which ran > > in basically the same time as his custom C code. Very impressive! > > Very nice! Care to share the code? Will do. I'll have to do some tidying up (at the moment, it's in an IPython notebook on my PC, full of chunks of code trying various approaches) but I'll post it in a few days. The C++ code isn't postable, unfortunately, as it's not open source. Also, it's not *quite* custom C++, but rather an expression you feed into his program that does this type of calculation. The program's built specifically for doing this type of problem, and runs an expression tree in a tight C++ loop (there's a small virtual function call overhead, but that's about it). More accurately, I guess, it's a custom C++ application for solving this class of problem. But regardless, I'd expected Python to only manage to be "good enough", not to equal the C++ application, so I'm still impressed! But it's a nice problem as an introduction to numpy - hard enough to be worth the effort, and it *needs* the performance numpy gives. But simple enough that solving it is achievable. Paul From ian.g.kelly at gmail.com Mon Feb 9 17:42:23 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 9 Feb 2015 15:42:23 -0700 Subject: Python 3 and the requests library In-Reply-To: References: <9b2a1b9d-4f1b-4786-a3a6-2d414074da6f@googlegroups.com> <96003963-8ca1-4b62-811d-05d6f6a5709d@googlegroups.com> <97b23ac0-c363-43d0-815e-198354d39c58@googlegroups.com> Message-ID: On Mon, Feb 9, 2015 at 2:37 PM, Brian wrote: > Thank you, Ian and Zack! That was exactly the issue. Apparently, having a token.py script (one of my first Python 2 scripts to get an authorization token from a billing server) is OK for Python 2 but breaks Python 3. Local modules that have the same absolute module path as standard library modules will cause problems in either version of Python. I think it's unfortunate that Python files that happen to live in the same directory as the main script automatically get treated as top-level modules that shadow the standard library. You're not the first person to be confused by this. It appears that the reason this works for you in Python 2 and not in Python 3 is because the linecache module doesn't import tokenize in Python 2, whereas it does in Python 3. If you had tried to import tokenize directly in Python 2 then I expect you'd have the same problem. From steve+comp.lang.python at pearwood.info Mon Feb 9 18:30:14 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 10:30:14 +1100 Subject: __next__ and StopIteration References: Message-ID: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> Charles Hixson wrote: > I'm trying to write a correct iteration over a doubly indexed container, > and what I've got so far is: def __next__ (self): > for row in range(self._rows): > for col in range(self._cols): > if self._grid[row][col]: > yield self._grid[row][col] > #end if > #end for col > #end for row > raise StopIteration That's wrong, you don't use yield in __next__ as that will turn it into a generator function. Here is a simplified example demonstrating the problem: py> class X(object): ... def __next__(self): ... yield 1 ... yield 2 ... py> x = X() py> next(x) py> next(x) py> next(x) py> next(x) The way you write iterators is like this: Method 1 (the hard way): - Give your class an __iter__ method which simply returns self: def __iter__(self): return self - Give your class a __next__ method (`next` in Python 2) which *returns* a value. You will need to track which value to return yourself. It must raise StopIteration when there are no more values to return. Don't use yield. def __next__(self): value = self.value if value is None: raise StopIteration self.value = self.calculate_the_next_value() return value Your class is itself an iterator. Method 2 (the easy way): - Don't write a __next__ method at all. - Give your class an __iter__ method which returns an iterator. E.g.: def __iter__(self): return iter(self.values) In this case, your class is not itself an iterator, but it is iterable: calling iter(myinstance) will return an iterator, which is enough. If you don't have a convenient collection of values to return, you can conveniently use a generator, yielding values you want and just falling off the end (or returning) when you are done. E.g.: def __iter__(self): while self.value is not None: yield self.value self.calculate_the_next_value() In your case, it looks to me that what you need is something like: def __iter__(self): for row in range(self._rows): for col in range(self._cols): if self._grid[row][col]: yield self._grid[row][col] which is probably better written as: # untested -- I may have the row/col order backwards def __iter__(self): for column in self._grid: for item in column: if value: yield value As a general rule, Python is not Fortran. If you find yourself wanting to write code that iterates over an index, then indexes into a list or array, 99.9% of the time you are better off just iterating over the list or array directly: # not this! for i in range(len(mylist)): value = mylist[i] print(value) # instead use this for value in mylist: print(value) If you need both the index and the list item, use enumerate: for i, value in enumerate(mylist): print(i, value) -- Steven From james8booker at hotmail.com Mon Feb 9 18:52:52 2015 From: james8booker at hotmail.com (james8booker at hotmail.com) Date: Mon, 9 Feb 2015 15:52:52 -0800 (PST) Subject: TypeError: list indices must be integers, not tuple Message-ID: import random RandomNum = random.randint(0,7) restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or Indian?") if restraunt == ("Pizza"): fav = ("1") elif restraunt == ("Chinease"): fav = ("2") elif restraunt == ("Indian"): fav = ("3") else: print("Try using a capital letter, eg; 'Chinease'") Menu = [["Barbeque pizza","Peparoni","Hawain"],["Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]] print Menu[fav,RandomNum] ^ TypeError: list indices must be integers, not tuple How do I set a variable to a random number then use it as a list indece, (I'm only a student in his first 6 months of using python) From ian.g.kelly at gmail.com Mon Feb 9 18:56:36 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 9 Feb 2015 16:56:36 -0700 Subject: __next__ and StopIteration In-Reply-To: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano wrote: > The way you write iterators is like this: > > > Method 1 (the hard way): > > - Give your class an __iter__ method which simply returns self: > > def __iter__(self): > return self > > - Give your class a __next__ method (`next` in Python 2) which *returns* a > value. You will need to track which value to return yourself. It must raise > StopIteration when there are no more values to return. Don't use yield. > > def __next__(self): > value = self.value > if value is None: > raise StopIteration > self.value = self.calculate_the_next_value() > return value > > Your class is itself an iterator. This is an anti-pattern, so don't even suggest it. Iterables should never be their own iterators. Otherwise, your iterable can only be iterated over once! The proper version of the "hard way" is: 1) The __iter__ method of the iterable constructs a new iterator instance and returns it. 2) The __iter__ method of the *iterator* simply returns itself. 3) The __next__ method of the iterator tracks the current value and returns the next value. Note that the iterator should never store the iterable's data internally, unless the iterable is immutable and the calculation is trivial (e.g. a range object). Instead, it should determine the next value by referring to its source iterable. From steve+comp.lang.python at pearwood.info Mon Feb 9 18:59:13 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 10:59:13 +1100 Subject: Please help. References: Message-ID: <54d949d3$0$12990$c3e8da3$5496439d@news.astraweb.com> Hello Chimex, and welcome. First, before I answer your question, I have a favour to ask. Please adjust your email program to send Plain Text as well as (or even instead of) so-called "Rich Text". When you send Rich Text, many people here will see your email like this: chimex60 at gmail.com wrote: > style="">
?Hello. Am trying to change the key words to my tribal > language. Eg change English language: print() to igbo language: de(). I > have been stuck for months I need a mentor or someone that can guide me > and answer some of my questions when I get stuck. > Thanks..
I will really appreciate it if someone attends > to me.
I hope you understand why most people won't bother to try to read or respond to such a mess. So-called "Rich Text" (actually HTML) bloats your email, making it bigger and slower than it needs to be, and it can be a security threat to the receiver (unscrupulous people can insert hostile code in the HTML, or web bugs, or other nasties). Or worse, it means that we're stuck with reading people's messages in some unspeakably horrible combination of ugly fonts, clashing colours and dancing fairies dancing across the page. So please understand that especially on a technical forum like this, probably 90% of of people will respond poorly to anything written in Rich Text alone. Moving on to your actual question: Am trying to change the key words to my tribal language. Eg change English language: print() to igbo language: de(). I know of two projects that do the same thing, ChinesePython and Teuton. http://www.chinesepython.org/ http://reganmian.net/blog/2008/11/21/chinese-python-translating-a-programming-language/ http://www.fiber-space.de/EasyExtend/doc/teuton/teuton.htm The Teuton page links to a post on Artima by Andy Dent which discusses this further. Basically, the idea is that you start with the source code to Python, written in C, change the keywords, and recompile. If nothing breaks, you should then be able to program using non-English keywords. As an alternative, you might consider using a pre-processor which translates your dialect of Python+Igbo to standard Python before running the code. Google for LikePython and LOLPython for some ideas. -- Steven From ethan at stoneleaf.us Mon Feb 9 19:02:31 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 09 Feb 2015 16:02:31 -0800 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: Message-ID: <54D94A97.6040902@stoneleaf.us> On 02/09/2015 03:52 PM, james8booker at hotmail.com wrote: > import random > RandomNum = random.randint(0,7) > restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or Indian?") > if restraunt == ("Pizza"): > fav = ("1") > > elif restraunt == ("Chinease"): > fav = ("2") > > elif restraunt == ("Indian"): > fav = ("3") > > else: > print("Try using a capital letter, eg; 'Chinease'") > > Menu = [["Barbeque pizza","Peparoni","Hawain"],["Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]] > > print Menu[fav,RandomNum] > ^ > TypeError: list indices must be integers, not tuple > > How do I set a variable to a random number then use it as a list indece, (I'm only a student in his first 6 months of using python) When you say Menu[fav,RandomNum] the `fav,RandomNum` portion is a tuple. `fav` should be 1 or 2 or 3, not "1" nor "2" nor "3". `RandomNum` should be be `random.randint(0,2)` Finally: submenu = Menu[fav] random_food = submenu[RandomNum] -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ryan.stuart.85 at gmail.com Mon Feb 9 19:05:54 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Tue, 10 Feb 2015 00:05:54 +0000 Subject: TypeError: list indices must be integers, not tuple References: Message-ID: Hi, There is a lot of issues with this code. First, setting fav to a 1 tuples with a string probably isn't what you want. What you probably mean is: if restraunt == ("Pizza"): fav = 1 Second, when you are trying to lookup items in Menu, you are using the incorrect fav. Lists have int indicies (just like the error points out). Values like ("1") aren't integers. Thirdly, Menu is a list of lists. To fetch "Barbeque pizza" from Menu, you need to do Menu[0][0], not Menu[0, 0]. Finally, Python comes with a style guide which you can find in pep8 . Your code violates that guide in many places. It might be worth working through the Python Tutorial . Cheers On Tue Feb 10 2015 at 9:55:40 AM wrote: > import random > RandomNum = random.randint(0,7) > restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or > Indian?") > if restraunt == ("Pizza"): > fav = ("1") > > elif restraunt == ("Chinease"): > fav = ("2") > > elif restraunt == ("Indian"): > fav = ("3") > > else: > print("Try using a capital letter, eg; 'Chinease'") > > Menu = [["Barbeque pizza","Peparoni","Hawain"],[" > Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]] > > print Menu[fav,RandomNum] > ^ > TypeError: list indices must be integers, not tuple > > How do I set a variable to a random number then use it as a list indece, > (I'm only a student in his first 6 months of using python) > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip.montanaro at gmail.com Mon Feb 9 19:36:51 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 9 Feb 2015 18:36:51 -0600 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 9, 2015 at 2:38 PM, Skip Montanaro wrote: > On Mon, Feb 9, 2015 at 2:05 PM, Zachary Ware > wrote: > > If all else fails, you can try ftfy to fix things: > > http://ftfy.readthedocs.org/en/latest/ > > Thanks for the pointer. I would prefer to not hand-mangle this stuff > in case I get another database dump from my USMS friends. Something > like ftfy should help things "just work". > And indeed it did. Thanks Zachary. Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Mon Feb 9 19:42:52 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 11:42:52 +1100 Subject: __next__ and StopIteration References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Ian Kelly wrote: > On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano > wrote: [...] >> Your class is itself an iterator. > > This is an anti-pattern, so don't even suggest it. Iterables should > never be their own iterators. Otherwise, your iterable can only be > iterated over once! Hmmm, good point. However, I will point out a couple of factors: Ultimately, that's the correct behaviour for *iterator* classes. With the rich set of tools available to build iterators from built-in parts, it is rare that you need to write your own class with a __next__ method, but if you do, that's the way you want it to behave. Whether that *iterator* class should be the same class as the *iterable* class is another story. In the built-ins, they mostly (always?) come in pairs: tuple <-> tuple_iterator list <-> list_iterator dict <-> dict_keyiterator set <-> set_iterator range <-> range_iterator so that's an excellent sign that doing so is best practice, but it should not be seen as *required*. After all, perhaps you have good reason for wanting your iterable class to only be iterated over once. Also, *technically* iterators may be re-iterable. The docs say that iterators which fail to raise StopIteration forever once they are exhausted are "broken", but the docs do not forbid broken iterators. Consenting adults and all that. You might want an iterator with a reset() method. Even an outright broken iterator! def __next__(self): if random.random() < 0.1: raise StopIteration return random.random() Why you would want one, I don't know, but if you have a hankering for such a beast, Python lets you do it. > The proper version of the "hard way" is: > > 1) The __iter__ method of the iterable constructs a new iterator > instance and returns it. > > 2) The __iter__ method of the *iterator* simply returns itself. > > 3) The __next__ method of the iterator tracks the current value and > returns the next value. Note that the iterator should never store the > iterable's data internally, unless the iterable is immutable and the > calculation is trivial (e.g. a range object). Instead, it should > determine the next value by referring to its source iterable. -- Steven From rosuav at gmail.com Mon Feb 9 19:54:22 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 11:54:22 +1100 Subject: __next__ and StopIteration In-Reply-To: <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 10, 2015 at 11:42 AM, Steven D'Aprano wrote: > Also, *technically* iterators may be re-iterable. The docs say that > iterators which fail to raise StopIteration forever once they are exhausted > are "broken", but the docs do not forbid broken iterators. Consenting > adults and all that. You might want an iterator with a reset() method. Even > an outright broken iterator! > > def __next__(self): > if random.random() < 0.1: raise StopIteration > return random.random() > > Why you would want one, I don't know, but if you have a hankering for such a > beast, Python lets you do it. Yes, it is allowed. But when you write code that's documented as being "broken", you should expect annoying, subtle errors, maybe a long way down the track. You know all those niggling problems you get when you find some NaNs in a series of numbers, and suddenly things don't sort stably and such? You'll hanker for those well-defined days. :) ChrisA From ckaynor at zindagigames.com Mon Feb 9 19:59:17 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Mon, 9 Feb 2015 16:59:17 -0800 Subject: __next__ and StopIteration In-Reply-To: <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano wrote: > so that's an excellent sign that doing so is best practice, but it should > not be seen as *required*. After all, perhaps you have good reason for > wanting your iterable class to only be iterated over once. In fact, there is one in the stdlib, the "file" object, which has a __iter__ which returns self. The code below shows this, There are a number of good reasons: perhaps there is no good way to reset the iterator or there is outside state that has to be managed. I'm also sure there are other reasons I cannot think of right now. > > Also, *technically* iterators may be re-iterable. The docs say that > iterators which fail to raise StopIteration forever once they are exhausted > are "broken", but the docs do not forbid broken iterators. Consenting > adults and all that. You might want an iterator with a reset() method. Even > an outright broken iterator! > > def __next__(self): > if random.random() < 0.1: raise StopIteration > return random.random() > > Why you would want one, I don't know, but if you have a hankering for such a > beast, Python lets you do it. The "file" object is also an example of this. It is technically a broken iterator according to the docs: Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> f = open("d:/test.txt") >>> iter(f) is f True >>> for l in f: ... print(l) ... line 1 line 2 line 3 >>> for l in f: ... print(l) ... >>> f.seek(0) 0 >>> for l in f: ... print(l) ... line 1 line 2 line 3 >>> next(f) Traceback (most recent call last): File "", line 1, in StopIteration >>> f.seek(0) 0 >>> next(f) # This should throw StopIteration as it has previously thrown StopIteration. 'line 1\n' >>> Chris From steve+comp.lang.python at pearwood.info Mon Feb 9 20:11:54 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 12:11:54 +1100 Subject: __next__ and StopIteration References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d95adc$0$13002$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Tue, Feb 10, 2015 at 11:42 AM, Steven D'Aprano > wrote: >> Also, *technically* iterators may be re-iterable. The docs say that >> iterators which fail to raise StopIteration forever once they are >> exhausted are "broken", but the docs do not forbid broken iterators. >> Consenting adults and all that. You might want an iterator with a reset() >> method. Even an outright broken iterator! >> >> def __next__(self): >> if random.random() < 0.1: raise StopIteration >> return random.random() >> >> Why you would want one, I don't know, but if you have a hankering for >> such a beast, Python lets you do it. > > Yes, it is allowed. But when you write code that's documented as being > "broken", you should expect annoying, subtle errors, maybe a long way > down the track. Well, that depends, don't it? If you're writing library code, you need to be much more careful and thoughtful about both your API and implementation. But if you're writing something quick and dirty for a one-off script, even a script that is going to be used in perpetuity, you can afford a lot more sloppiness. Better something sloppy that works today than something perfect next year. Say my script just outputs a bunch of random numbers, one per line, and I pipe the output to a file in the shell: mkrnd.py > the_numbers.txt Am I really going to care if the iterator used to generate those random numbers is technically "broken"? Probably not. And if I do, some time in the distant future, oh well, I'll "fix" it then. -- Steven From ned at nedbatchelder.com Mon Feb 9 20:30:21 2015 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 09 Feb 2015 20:30:21 -0500 Subject: __next__ and StopIteration In-Reply-To: References: <54D9072F.3010904@earthlink.net> Message-ID: On 2/9/15 2:24 PM, Ned Batchelder wrote: > On 2/9/15 2:14 PM, Charles Hixson wrote: >> I'm trying to write a correct iteration over a doubly indexed container, >> and what I've got so far is: def __next__ (self): >> for row in range(self._rows): >> for col in range(self._cols): >> if self._grid[row][col]: >> yield self._grid[row][col] >> #end if >> #end for col >> #end for row >> raise StopIteration >> >> What bothers me is that it doesn't look like it would continue to raise >> StopIteration if it were called again, which is what >> https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says >> is correct. How should this be fixed? > > You are using yield, which means __next__ is a generator. As such, you > don't have to explicitly raise StopIteration at all. Just remove that > statement, and you should be fine. > > Also, look into how you are posting, the code is nearly mangled. :( > Oops, __next__ isn't right here, it should be __iter__: def __iter__(self): for row in range(self._rows): for col in range(self._cols): if self._grid[row][col]: yield self._grid[row][col] -- Ned Batchelder, http://nedbatchelder.com From rosuav at gmail.com Mon Feb 9 20:58:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 12:58:44 +1100 Subject: __next__ and StopIteration In-Reply-To: <54d95adc$0$13002$c3e8da3$5496439d@news.astraweb.com> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> <54d95adc$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 10, 2015 at 12:11 PM, Steven D'Aprano wrote: >> Yes, it is allowed. But when you write code that's documented as being >> "broken", you should expect annoying, subtle errors, maybe a long way >> down the track. > > Well, that depends, don't it? > > Am I really going to care if the iterator used to generate those random > numbers is technically "broken"? Probably not. And if I do, some time in > the distant future, oh well, I'll "fix" it then. If you inhale asbestos, you should expect annoying, maybe subtle, errors, maybe a long way down the track. Good news is, the lab boys say the symptoms of asbestos poisoning show a median latency of 44.6 years, so if you're thirty or older, you're laughing. Worst case scenario, you miss out on a few rounds of canasta. Do you care that your lungs are technically "broken"? Probably not. It's something to be aware of when you consider working in certain environments, but "be aware of" doesn't mean "absolutely always avoid". Okay, maybe it does with asbestos... but not with broken iterators. :) ChrisA From i at introo.me Mon Feb 9 21:52:43 2015 From: i at introo.me (Shiyao Ma) Date: Tue, 10 Feb 2015 10:52:43 +0800 Subject: Weird behavior on __dict__ attr Message-ID: <20150210025243.GB1266@FuzzyHorror> Hi. My context is a little hard to reproduce. NS3 is a network simulation tool written in C++. I am using its Python binding. So the class I am dealing with is from a .so file. Say, I do the following: % import ns.network.Node as Node # Node is a class # it has a __dict__ attr # Now I instantiate an instance of Node n = Node() # I checked, there is no __dict__ on 'n' # but the following succeeds. n.foobar = 3 My understanding is the foobar is stored in n.__dict__, but seemingly n has no __dict__. So where does the foobar go? TIA. -- Shiyao Ma http://introo.me From davea at davea.name Mon Feb 9 22:05:11 2015 From: davea at davea.name (Dave Angel) Date: Mon, 09 Feb 2015 22:05:11 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: <54D94A97.6040902@stoneleaf.us> References: <54D94A97.6040902@stoneleaf.us> Message-ID: <54D97567.5060605@davea.name> On 02/09/2015 07:02 PM, Ethan Furman wrote: > On 02/09/2015 03:52 PM, james8booker at hotmail.com wrote: >> import random >> RandomNum = random.randint(0,7) >> restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or Indian?") >> if restraunt == ("Pizza"): >> fav = ("1") >> >> elif restraunt == ("Chinease"): >> fav = ("2") >> >> elif restraunt == ("Indian"): >> fav = ("3") >> >> else: >> print("Try using a capital letter, eg; 'Chinease'") >> So just what is RandomNum supposed to represent? You've selected it from an interval of 0 to 7, but you don't have 8 of anything. The most logical possibility I can figure is you want to use it instead of whatever the user has typed into your raw input. Like in the else clause. If that's the case, you'd want to add a fav = RandomNum line in the else clause. Of course, as Ethan has pointed out, all the other assignments to fav want to be integer, not string. You can't use a string to index a lis. >> Menu = [["Barbeque pizza","Peparoni","Hawain"],["Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]] >> >> print Menu[fav,RandomNum] Now that you've got a single value for fav, just say print Menu[fav] to print the submenu. Now if Ethan has guessed right, that you wanted the random value to choose from the submenu, then you would/should have created it after you have the submenu, so you know how many possibilities there are. Something like (untested): RandomNum = random.randint(0, len(submenu)-1) -- DaveA From davea at davea.name Mon Feb 9 22:12:01 2015 From: davea at davea.name (Dave Angel) Date: Mon, 09 Feb 2015 22:12:01 -0500 Subject: Weird behavior on __dict__ attr In-Reply-To: <20150210025243.GB1266@FuzzyHorror> References: <20150210025243.GB1266@FuzzyHorror> Message-ID: <54D97701.3080803@davea.name> On 02/09/2015 09:52 PM, Shiyao Ma wrote: > Hi. > > My context is a little hard to reproduce. WHY don't you try? Telling us about a class without showing how it's defined leaves us all guessing. Start by telling us Python version. And if it's 2.x, tell us whether this class is an old style or new style class. > > NS3 is a network simulation tool written in C++. I am using its Python binding. > > So the class I am dealing with is from a .so file. > > Say, I do the following: > > % > > import ns.network.Node as Node > > # Node is a class > # it has a __dict__ attr > > # Now I instantiate an instance of Node > n = Node() > > # I checked, there is no __dict__ on 'n' > # but the following succeeds. > > n.foobar = 3 > > > > My understanding is the foobar is stored in n.__dict__, but seemingly n has no __dict__. > > So where does the foobar go? > Lots of possibilities. Simplest is slots. if you define __slots__, then there's no dictionary in each instance. https://docs.python.org/3.4/reference/datamodel.html#slots -- DaveA From charleshixsn at earthlink.net Mon Feb 9 23:33:57 2015 From: charleshixsn at earthlink.net (Charles Hixson) Date: Mon, 09 Feb 2015 20:33:57 -0800 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54D98A35.9060203@earthlink.net> On 02/09/2015 03:56 PM, Ian Kelly wrote: > On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano > wrote: >> The way you write iterators is like this: >> >> >> Method 1 (the hard way): >> >> - Give your class an __iter__ method which simply returns self: >> >> def __iter__(self): >> return self >> >> - Give your class a __next__ method (`next` in Python 2) which *returns* a >> value. You will need to track which value to return yourself. It must raise >> StopIteration when there are no more values to return. Don't use yield. >> >> def __next__(self): >> value = self.value >> if value is None: >> raise StopIteration >> self.value = self.calculate_the_next_value() >> return value >> >> Your class is itself an iterator. > This is an anti-pattern, so don't even suggest it. Iterables should > never be their own iterators. Otherwise, your iterable can only be > iterated over once! > > The proper version of the "hard way" is: > > 1) The __iter__ method of the iterable constructs a new iterator > instance and returns it. > > 2) The __iter__ method of the *iterator* simply returns itself. > > 3) The __next__ method of the iterator tracks the current value and > returns the next value. Note that the iterator should never store the > iterable's data internally, unless the iterable is immutable and the > calculation is trivial (e.g. a range object). Instead, it should > determine the next value by referring to its source iterable. So if I'm understanding this correctly, I should implement as an internal class within Grid something like: class GridIter(Iterator): """ A class to iterate over the cells of a Grid instance Vars: row :: The row currently being iterated over. col :: Yhe column currently being iterated over. grid :: The grid instance being iterated over. """ def __init__ (self, grid): """ Params: grid :: The instance of the grid to iterate over. """ self.row = -1 self.col = -1 self.grid = grid #end __init__ def __iter__ (self): return self def __next__ (self): if self.row == -1 and self.col == -1: self.row = 0 self.col = 0 elif self.col >= grid.cols(): self.row = self.row + 1 self.col = 0 if self.row > grid.rows(): raise StopIteration if not [row, col] in self.grid: return self.__next__() return grid(row, col) From rosuav at gmail.com Mon Feb 9 23:46:25 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 15:46:25 +1100 Subject: __next__ and StopIteration In-Reply-To: <54D98A35.9060203@earthlink.net> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> Message-ID: On Tue, Feb 10, 2015 at 3:33 PM, Charles Hixson wrote: >> The proper version of the "hard way" is: >> >> 1) The __iter__ method of the iterable constructs a new iterator >> instance and returns it. >> >> 2) The __iter__ method of the *iterator* simply returns itself. >> >> 3) The __next__ method of the iterator tracks the current value and >> returns the next value. Note that the iterator should never store the >> iterable's data internally, unless the iterable is immutable and the >> calculation is trivial (e.g. a range object). Instead, it should >> determine the next value by referring to its source iterable. > > So if I'm understanding this correctly, I should implement as an internal > class within Grid something like: > class GridIter(Iterator): Apart from the fact that you shouldn't have to explicitly subclass Iterator, yes. But this is the hard way to do things. The easy way is to simply define an __iter__ method on your Grid which returns an iterator - and one excellent form of iterator, for custom classes like this, is a generator object. Your original code can slot happily in, with one tiny change: class Grid: blah blah def __iter__(self): for row in range(self._rows): for col in range(self._cols): if self._grid[row][col]: yield self._grid[row][col] The only change is to remove the explicit StopIteration at the end; once your generator function terminates, the generator object will raise StopIteration forever afterward, without any help from you. (Also, post-PEP479, the explicit raise will actually cause RuntimeError, so it's not just superfluous, but actually a problem.) But I'm guessing that your grid and rows are actually iterable themselves. If they are, you can cut the code down to this: def __iter__(self): for row in self._grid: for cell in row: if cell: yield cell or a generator expression: def __iter__(self): return (cell for row in self._grid for cell in row if cell) or itertools.chain and filter, if you so desired. As long as you return an iterator, you're fine. This is far and away the easiest way to make a class iterable. ChrisA From davea at davea.name Mon Feb 9 23:48:14 2015 From: davea at davea.name (Dave Angel) Date: Mon, 09 Feb 2015 23:48:14 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: <54D97567.5060605@davea.name> References: <54D94A97.6040902@stoneleaf.us> <54D97567.5060605@davea.name> Message-ID: <54D98D8E.3050108@davea.name> On 02/09/2015 10:05 PM, Dave Angel wrote: > On 02/09/2015 07:02 PM, Ethan Furman wrote: >> On 02/09/2015 03:52 PM, james8booker at hotmail.com wrote: >>> import random >>> RandomNum = random.randint(0,7) >>> restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease >>> or Indian?") >>> if restraunt == ("Pizza"): >>> fav = ("1") >>> >>> elif restraunt == ("Chinease"): >>> fav = ("2") >>> >>> elif restraunt == ("Indian"): >>> fav = ("3") >>> >>> else: >>> print("Try using a capital letter, eg; 'Chinease'") >>> > > So just what is RandomNum supposed to represent? You've selected it > from an interval of 0 to 7, but you don't have 8 of anything. The most > logical possibility I can figure is you want to use it instead of > whatever the user has typed into your raw input. Like in the else > clause. If that's the case, you'd want to add a > fav = RandomNum > line in the else clause. > > Of course, as Ethan has pointed out, all the other assignments to fav > want to be integer, not string. You can't use a string to index a lis. > >>> Menu = [["Barbeque >>> pizza","Peparoni","Hawain"],["Curry","Noodles","Rice"],["Tika >>> Masala","Special Rice","Onion Bargees"]] >>> >>> print Menu[fav,RandomNum] > > Now that you've got a single value for fav, just say > print Menu[fav] > to print the submenu. > > Now if Ethan has guessed right, that you wanted the random value to > choose from the submenu, then you would/should have created it after you > have the submenu, so you know how many possibilities there are. > > > Something like (untested): > RandomNum = random.randint(0, len(submenu)-1) > > > Perhaps it's worth suggesting that you use random.choice() instead, and use it directly on the sublist. If you also make your data structure a dict of lists, then the whole thing becomes very simple. (untested) Menu = { "Pizza" : ["Barbeque pizza","Peparoni","Hawain"], "Chinease" : ["Curry","Noodles","Rice"],["Tika Masala", "Indian" : "Special Rice","Onion Bargees"] } restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or Indian?") submenu = menu[restraunt] #you might want some error handling, in case they get it wrong fooditem = random.choice(submenu) print fooditem -- DaveA From steve+comp.lang.python at pearwood.info Tue Feb 10 00:54:14 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 16:54:14 +1100 Subject: __next__ and StopIteration References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54d99d07$0$28208$c3e8da3$76491128@news.astraweb.com> Chris Kaynor wrote: > On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano > wrote: >> so that's an excellent sign that doing so is best practice, but it should >> not be seen as *required*. After all, perhaps you have good reason for >> wanting your iterable class to only be iterated over once. > > In fact, there is one in the stdlib, the "file" object, which has a > __iter__ which returns self. The code below shows this, [...] > The "file" object is also an example of this. It is technically a > broken iterator according to the docs: Awesome example! -- Steve From tjreedy at udel.edu Tue Feb 10 00:57:05 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 10 Feb 2015 00:57:05 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: Message-ID: On 2/9/2015 6:52 PM, james8booker at hotmail.com wrote: > import random > RandomNum = random.randint(0,7) > restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or Indian?") > if restraunt == ("Pizza"): > fav = ("1") As a style note, putting parentheses around strings is worse than useless. > elif restraunt == ("Chinease"): > fav = ("2") > > elif restraunt == ("Indian"): > fav = ("3") > > else: > print("Try using a capital letter, eg; 'Chinease'") > > Menu = [["Barbeque pizza","Peparoni","Hawain"],["Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]] > > print Menu[fav,RandomNum] > ^ > TypeError: list indices must be integers, not tuple > > How do I set a variable to a random number then use it as a list indece, (I'm only a student in his first 6 months of using python) > -- Terry Jan Reedy From charleshixsn at earthlink.net Tue Feb 10 01:16:14 2015 From: charleshixsn at earthlink.net (Charles Hixson) Date: Mon, 09 Feb 2015 22:16:14 -0800 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> Message-ID: <54D9A22E.2070708@earthlink.net> On 02/09/2015 08:46 PM, Chris Angelico wrote: > On Tue, Feb 10, 2015 at 3:33 PM, Charles Hixson > wrote: >>> The proper version of the "hard way" is: >>> >>> 1) The __iter__ method of the iterable constructs a new iterator >>> instance and returns it. >>> >>> 2) The __iter__ method of the *iterator* simply returns itself. >>> >>> 3) The __next__ method of the iterator tracks the current value and >>> returns the next value. Note that the iterator should never store the >>> iterable's data internally, unless the iterable is immutable and the >>> calculation is trivial (e.g. a range object). Instead, it should >>> determine the next value by referring to its source iterable. >> So if I'm understanding this correctly, I should implement as an internal >> class within Grid something like: >> class GridIter(Iterator): > Apart from the fact that you shouldn't have to explicitly subclass > Iterator, yes. But this is the hard way to do things. The easy way is > to simply define an __iter__ method on your Grid which returns an > iterator - and one excellent form of iterator, for custom classes like > this, is a generator object. Your original code can slot happily in, > with one tiny change: > > class Grid: > blah blah > > def __iter__(self): > for row in range(self._rows): > for col in range(self._cols): > if self._grid[row][col]: > yield self._grid[row][col] > > The only change is to remove the explicit StopIteration at the end; > once your generator function terminates, the generator object will > raise StopIteration forever afterward, without any help from you. > (Also, post-PEP479, the explicit raise will actually cause > RuntimeError, so it's not just superfluous, but actually a problem.) > > But I'm guessing that your grid and rows are actually iterable > themselves. If they are, you can cut the code down to this: > > def __iter__(self): > for row in self._grid: > for cell in row: > if cell: yield cell > > or a generator expression: > > def __iter__(self): > return (cell for row in self._grid for cell in row if cell) > > or itertools.chain and filter, if you so desired. As long as you > return an iterator, you're fine. This is far and away the easiest way > to make a class iterable. > > ChrisA Yes, rows and cols are lists, but I'm going to need to iterate through them more than once. I'd rather do without a included class, but if a properly formed iterator can only be cycled through once, and if I understand properly that means I can't use the "class instance is it's own iterator" form. From rosuav at gmail.com Tue Feb 10 01:38:52 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 17:38:52 +1100 Subject: __next__ and StopIteration In-Reply-To: <54D9A22E.2070708@earthlink.net> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> <54D9A22E.2070708@earthlink.net> Message-ID: On Tue, Feb 10, 2015 at 5:16 PM, Charles Hixson wrote: > Yes, rows and cols are lists, but I'm going to need to iterate through them > more than once. I'd rather do without a included class, but if a properly > formed iterator can only be cycled through once, and if I understand > properly that means I can't use the "class instance is it's own iterator" > form. When you write __iter__ as a generator function, what you actually have is a factory for iterator objects. (Generators are iterators.) When something calls __iter__ on your Grid, it gets back a brand new iterator which is independent of every other iterator from that or any other Grid, so you can iterate over the same Grid more than once. You're not writing __next__ here, so you're not using the class instance as its own iterator, which means that that consideration doesn't apply. ChrisA From torque.india at gmail.com Tue Feb 10 02:30:04 2015 From: torque.india at gmail.com (OmPs) Date: Tue, 10 Feb 2015 13:00:04 +0530 Subject: Varable parsing error with python Message-ID: In an XML file which contain a tag and then these tags contain properties, which later are being called in anoter tag where these properties are expanded as variables. For eg. 1.0 1.2 Red Hat Enterprise Linux Server 5 X86_64 testsapppath my testapp area ${APPVERSION} So now i have written a python program whcih is parsing this program, so what i am doing in here is reading this xml file with open(filename, "r") as f: orig_xml = f.read() #unescape html characters like %20 xmlstring = urllib.unquote(orig_xml) #convert xml to dict doc = xmltodict.parse(xmlstring) #replace property values in xml string via instancename s = Template(xmlstring) #iterate over instances/ match the one we need for i in doc.get("application").get("instance", None): #when found # s/r properties in xml and then convert it back to dict try: if i.get("@name", None) == instancename: try: instance_property = dict(i.get("property")) final_string = s.safe_substitute(instance_property) final_dict = xmltodict.parse(final_string) except TypeError: final_dict = doc # Handle strings in case of dict, strings do not have get method. except AttributeError: final_dict = doc and when i am trying to get the version no. using a function for getting the package version def _getPackgeVersion(xmlfile, p): package = str(p) if isinstance(fpmdict["application"]["package"], list): for i in fpmdict["application"]["package"]: if i["@name"] == p: _pkgVersion = i["version"] else: _pkgversion = fpmdict["application"]["package"]["version"] return _pkgVersion pkgVersion = _getPackgeVersion(final_doc, testpackage) print type(pkgVersion) I am getting the below error -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Feb 10 02:39:42 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 18:39:42 +1100 Subject: Varable parsing error with python In-Reply-To: References: Message-ID: On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: > def _getPackgeVersion(xmlfile, p): > package = str(p) > if isinstance(fpmdict["application"]["package"], list): > for i in fpmdict["application"]["package"]: > if i["@name"] == p: > _pkgVersion = i["version"] > else: > _pkgversion = fpmdict["application"]["package"]["version"] > return _pkgVersion One of your branches doesn't have a return statement in it, so Python just returns None. You may want to unindent that return statement one level. ChrisA From steve+comp.lang.python at pearwood.info Tue Feb 10 03:00:33 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 10 Feb 2015 19:00:33 +1100 Subject: Weird behavior on __dict__ attr References: Message-ID: <54d9baa2$0$2890$c3e8da3$76491128@news.astraweb.com> Shiyao Ma wrote: > Hi. > > My context is a little hard to reproduce. > > NS3 is a network simulation tool written in C++. I am using its Python > binding. > > So the class I am dealing with is from a .so file. So it is written in (probably) C and you don't have source code for it. > Say, I do the following: > > % > > import ns.network.Node as Node > > # Node is a class > # it has a __dict__ attr > > # Now I instantiate an instance of Node > n = Node() > > # I checked, there is no __dict__ on 'n' > # but the following succeeds. > > n.foobar = 3 It is hard to say exactly what is happening without source code. If Node is an extension class written in C, it could do almost anything. It certainly doesn't need to follow Python conventions for Python classes. If this were a pure-Python class, I can think of at least three ways to get this behaviour: - use __slots__ so that the class has a fixed set of members with no __dict__ used or needed; - use a custom descriptor - use a custom metaclass that changes the behaviour of lookups; I presume that extension classes written in C can do similar things. > My understanding is the foobar is stored in n.__dict__, but seemingly n > has no __dict__. > > So where does the foobar go? Here are some diagnostic tools you can use to try to determine what is going on: print(vars(n)) print(dir(n)) print(n.__slots__) o = Node.__dict__['foobar'] print(type(o), repr(o)) -- Steve From rosuav at gmail.com Tue Feb 10 03:13:46 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Feb 2015 19:13:46 +1100 Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: <54D8F438.1060402@case.edu> Message-ID: On Tue, Feb 10, 2015 at 5:52 AM, Skip Montanaro wrote: > > This snapshot was taken against a running LibreOffice instance here at work (on Linux). It would appear the fancy schmancy apostrophe was hosed up before the data ever got to me. Had a guy here with Windows pop up the original file I got in an actual Excel instance. Same bogosity. > Just what you always want to see, messy data! > Knowing that, I don't feel the least bit timid about just editing the darn CSV file to correct the encode/decode/encode error before loading the data into SQLite. I was worried that since I was doing some Python 3.x stuff involving Unicode for the first time that I'd screwed something up. > Py3 makes things so easy that the chances are you got it right :) As long as the encoding parameter on the open() call matches the file's actual encoding, chances are you're fine - everything else is done with Unicode characters. ChrisA From wxjmfauth at gmail.com Tue Feb 10 03:23:33 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 10 Feb 2015 00:23:33 -0800 (PST) Subject: Python 3.x stuffing utf-8 into SQLite db In-Reply-To: References: <54d83ced$0$12895$c3e8da3$5496439d@news.astraweb.com> Message-ID: <7ce36cdc-2cb6-41f9-964d-227248c9d33c@googlegroups.com> Le mardi 10 f?vrier 2015 01:37:15 UTC+1, Skip Montanaro a ?crit : > On Mon, Feb 9, 2015 at 2:38 PM, Skip Montanaro wrote: > On Mon, Feb 9, 2015 at 2:05 PM, Zachary Ware > > wrote: > > > If all else fails, you can try ftfy to fix things: > > > http://ftfy.readthedocs.org/en/latest/ > > > > Thanks for the pointer. I would prefer to not hand-mangle this stuff > > in case I get another database dump from my USMS friends. Something > > like ftfy should help things "just work". > > > > And indeed it did. Thanks Zachary. > %%%%%% ftfy: a mountain of absurdities. On top of this: ~buggy. Everything works fine if it's done correctly. There is nothing to fix. I have the feeling you are destroying a correct data file, and later you try to correct what you have destroyed. Basically the same experiment from Matthew Ruffalo: Office suite --> csv file saved as pd.txt. >From my GUI interactive interpreter (py32). >>> with open('pd.csv', encoding='utf-8') as f: ... r = f.read() ... >>> print(r) "Patrick's Day A1","Patrick's Day B1","Patrick's Day C1" "Patrick's Day A2","Patrick's Day C2","Patrick's Day C2" >>> Now what may happen, is that the terminal (the host system) may not display all these chars correctly (Windows, Russion *x, ...). In that case, one has to code correctly (Windows, Russion *x, ...) Still with the same GUI interpreter: >>> sys.stdout.sethostencoding('cp850') >>> outenc = sys.stdout.encoding >>> print(r.encode(outenc, 'replace').decode(outenc)) "Patrick?s Day A1","Patrick?s Day B1","Patrick?s Day C1" "Patrick?s Day A2","Patrick?s Day C2","Patrick?s Day C2" >>> sys.stdout.sethostencoding('iso-8859-5') >>> outenc = sys.stdout.encoding >>> print(r.encode(outenc, 'replace').decode(outenc)) "Patrick?s Day A1","Patrick?s Day B1","Patrick?s Day C1" "Patrick?s Day A2","Patrick?s Day C2","Patrick?s Day C2" This is exactly what can be observed in a web browser. Just for the fun, in fact a no-op. >>> sys.stdout.sethostencoding('utf-32-le') >>> outenc = sys.stdout.encoding >>> print(r.encode(outenc, 'replace').decode(outenc)) "Patrick's Day A1","Patrick's Day B1","Patrick's Day C1" "Patrick's Day A2","Patrick's Day C2","Patrick's Day C2" From torque.india at gmail.com Tue Feb 10 03:29:23 2015 From: torque.india at gmail.com (OmPs) Date: Tue, 10 Feb 2015 13:59:23 +0530 Subject: Varable parsing error with python In-Reply-To: References: Message-ID: On 10 Feb 2015 13:12, "Chris Angelico" wrote: > > On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: > > def _getPackgeVersion(xmlfile, p): > > package = str(p) > > if isinstance(fpmdict["application"]["package"], list): > > for i in fpmdict["application"]["package"]: > > if i["@name"] == p: > > _pkgVersion = i["version"] > > else: > > _pkgversion = fpmdict["application"]["package"]["version"] > > return _pkgVersion > > One of your branches doesn't have a return statement in it, so Python > just returns None. You may want to unindent that return statement one > level. > Tried that as well getting the same error. > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.g.kelly at gmail.com Tue Feb 10 03:30:24 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 10 Feb 2015 01:30:24 -0700 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 9, 2015 at 5:59 PM, Chris Kaynor wrote: > On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano > wrote: >> so that's an excellent sign that doing so is best practice, but it should >> not be seen as *required*. After all, perhaps you have good reason for >> wanting your iterable class to only be iterated over once. > > In fact, there is one in the stdlib, the "file" object, which has a > __iter__ which returns self. The code below shows this, Fair point. I suppose that's because the file paradigm itself has a concept of current position that can't easily be abstracted away. > The "file" object is also an example of this. It is technically a > broken iterator according to the docs: > > Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 > 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> f = open("d:/test.txt") >>>> iter(f) is f > True >>>> for l in f: > ... print(l) > ... > line 1 > > line 2 > > line 3 > >>>> for l in f: > ... print(l) > ... >>>> f.seek(0) > 0 >>>> for l in f: > ... print(l) > ... > line 1 > > line 2 > > line 3 > >>>> next(f) > Traceback (most recent call last): > File "", line 1, in > StopIteration >>>> f.seek(0) > 0 >>>> next(f) # This should throw StopIteration as it has previously thrown StopIteration. > 'line 1\n' IIRC the docs also warn that mixing file iteration with other file methods results in undefined or broken behavior. Maybe that's only for Python 2 files, however. From torque.india at gmail.com Tue Feb 10 03:30:27 2015 From: torque.india at gmail.com (OmPs) Date: Tue, 10 Feb 2015 14:00:27 +0530 Subject: Varable parsing error with python In-Reply-To: References: Message-ID: On 10 Feb 2015 13:59, "OmPs" wrote: > > > On 10 Feb 2015 13:12, "Chris Angelico" wrote: > > > > On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: > > > def _getPackgeVersion(xmlfile, p): > > > package = str(p) > > > if isinstance(fpmdict["application"]["package"], list): > > > for i in fpmdict["application"]["package"]: > > > if i["@name"] == p: > > > _pkgVersion = i["version"] > > > else: > > > _pkgversion = fpmdict["application"]["package"]["version"] > > > return _pkgVersion > > > > One of your branches doesn't have a return statement in it, so Python > > just returns None. You may want to unindent that return statement one > > level. > > > Tried that as well getting the same error. I feel its something to do with variable substitution. > > ChrisA > > -- > > https://mail.python.org/mailman/listinfo/python-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Tue Feb 10 06:35:23 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 10 Feb 2015 11:35:23 +0000 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: Message-ID: On 10/02/2015 00:05, Ryan Stuart wrote: > Hi, > > There is a lot of issues with this code. First, setting fav to a 1 > tuples with a string probably isn't what you want. What you probably > mean is: > > if restraunt == ("Pizza"): > fav = 1 > > Second, when you are trying to lookup items in Menu, you are using the > incorrect fav. Lists have int indicies (just like the error points out). > Values like ("1") aren't integers. > > Thirdly, Menu is a list of lists. To fetch "Barbeque pizza" from Menu, > you need to do Menu[0][0], not Menu[0, 0]. > > Finally, Python comes with a style guide which you can find in pep8 > . Your code violates that > guide in many places. It might be worth working through the Python > Tutorial . > > Cheers > > On Tue Feb 10 2015 at 9:55:40 AM > wrote: > > import random > RandomNum = random.randint(0,7) > restraunt = raw_input("What's your favourite takeaway?Pizza, > Chinease or Indian?") > if restraunt == ("Pizza"): > fav = ("1") > > elif restraunt == ("Chinease"): > fav = ("2") > > elif restraunt == ("Indian"): > fav = ("3") > > else: > print("Try using a capital letter, eg; 'Chinease'") > > Menu = [["Barbeque > pizza","Peparoni","Hawain"],["__Curry","Noodles","Rice"],["__Tika > Masala","Special Rice","Onion Bargees"]] > > print Menu[fav,RandomNum] > ^ > TypeError: list indices must be integers, not tuple > > How do I set a variable to a random number then use it as a list > indece, (I'm only a student in his first 6 months of using python) > -- > https://mail.python.org/__mailman/listinfo/python-list > > If you can show me a one tuple anywhere in the original code I'll happily buy you a tipple of your choice. Also please don't top post here, it makes following long threads difficult if not impossible to follow, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rustompmody at gmail.com Tue Feb 10 06:40:15 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 03:40:15 -0800 (PST) Subject: doc buglets? In-Reply-To: References: Message-ID: <6054f695-3528-49d0-9633-66c33472c660@googlegroups.com> On Monday, February 9, 2015 at 9:25:45 PM UTC+5:30, Mark Lawrence wrote: > On 09/02/2015 11:28, Rustom Mody wrote: > > Poking around in help() (python 3.4.2) I find > > > > * PACKAGES > > does not seem to have anything on packages > > * DYNAMICFEATURES > > seems to be some kind of footnote > > * SPECIALATTRIBUTES > > has 'bases' and 'subclasses'. It seems to me a more consistent naming > > for OOP would be in order. These are the OOP-metaphors I am familiar with > > - superclass ? subclass > > - base class ? derived class > > - parent class ? child class > > > > [Yeah if this is a bug its more than a docbug. > > So no, I am not suggesting changing the actual attribute names which would be a > > breaking change, just the stuff in help() > > ] > > > > So raise an issue and attach a patch. Sure ? can do. However for the first two I am not sure what the intentions of those entries are And for the third, I am not clear that anyone other than me finds the mismatching metaphors galling. From rustompmody at gmail.com Tue Feb 10 06:59:54 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 03:59:54 -0800 (PST) Subject: Incompatible idioms: relative imports, top-level program file In-Reply-To: References: <85lhkag1ts.fsf@benfinney.id.au> <85h9uyfy3v.fsf@benfinney.id.au> <54D54C90.2070008@stoneleaf.us> <858ugaft34.fsf_-_@benfinney.id.au> <54D5668C.9040600@stoneleaf.us> <084ad65b-8458-41d4-97bd-d5576d3a751b@googlegroups.com> <320f55ef-7ea2-4e41-9cc8-94a68326ca35@googlegroups.com> Message-ID: <85b6ec89-d97e-4c11-87b7-17882f431d46@googlegroups.com> On Sunday, February 8, 2015 at 3:52:19 AM UTC+5:30, Gregory Ewing wrote: > Rustom Mody wrote: > > Wanted to try out sympy. > > apt-install promised ? GB download, ? GB space usage > > > > Just getting a src-tarball was: 6M download, 30M after opening the tar. > > Have you actually tried compiling and using that > tarball, though? > > Sympy hooks into a lot of other libraries that > are themselves quite large. Apt-get was probably > planning to download and install all of them, > hence the large download size. > > If you tried to install from source, you would > likely have to install all those dependencies > yourself, or configure it to skip the ones you > don't want if it allows that. > > -- > Greg Yeah... Sympy output as on display http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-5-Sympy.ipynb looks quite beautiful and probably needs all those Tex etc fonts As it happens I did not need that I just wanted to demo some differences between object-level and meta-level . I guess the best choice for that would be lisp. I could not figure racket out ? terminologies seem to have changed in the last couple of decades ? so I took sympy's distinction of python-variable and math-variable as a good enough exemplar of meta-level and object-level. [Python's namespaces are not really firstclass like (some) scheme's are -- eg locals() makes a copy] Anyway? My point was not that particular use-case but the general point that one may want to play around with alternate packaging systems From davea at davea.name Tue Feb 10 09:28:03 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 09:28:03 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: Message-ID: <54DA1573.1030804@davea.name> On 02/10/2015 06:35 AM, Mark Lawrence wrote: > On 10/02/2015 00:05, Ryan Stuart wrote: >> Hi, >> > > If you can show me a one tuple anywhere in the original code I'll > happily buy you a tipple of your choice. print Menu[fav,RandomNum] was in the original code -- DaveA From breamoreboy at yahoo.co.uk Tue Feb 10 09:32:04 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 10 Feb 2015 14:32:04 +0000 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: <54DA1573.1030804@davea.name> References: <54DA1573.1030804@davea.name> Message-ID: On 10/02/2015 14:28, Dave Angel wrote: > On 02/10/2015 06:35 AM, Mark Lawrence wrote: >> On 10/02/2015 00:05, Ryan Stuart wrote: >>> Hi, >>> > >> >> If you can show me a one tuple anywhere in the original code I'll >> happily buy you a tipple of your choice. > > print Menu[fav,RandomNum] > > was in the original code > > > Thanks for the correction, I should have been more careful how I phrased my reply :( -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Tue Feb 10 09:33:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 01:33:38 +1100 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: <54DA1573.1030804@davea.name> References: <54DA1573.1030804@davea.name> Message-ID: On Wed, Feb 11, 2015 at 1:28 AM, Dave Angel wrote: >> If you can show me a one tuple anywhere in the original code I'll >> happily buy you a tipple of your choice. > > > print Menu[fav,RandomNum] > > was in the original code That's not a one-tuple (as in, a tuple with one element), it's a two-element tuple. The point is that ("3") doesn't create a tuple, it's just superfluous parentheses. ChrisA From lac at openend.se Tue Feb 10 09:36:41 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 10 Feb 2015 15:36:41 +0100 Subject: tabs and the Python3 console Message-ID: <201502101436.t1AEafIH032349@fido.openend.se> I have the debian version of python3 installed here. Python 3.4.2 (default, Nov 13 2014, 07:01:52) [GCC 4.9.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> But I cannot seem to type a tab here: >>> def fn(**kw): ... (I type a tab here, and get a beep. If I type a tab again I get: Display all 178 possibilities? (y or n) ArithmeticError( chr( AssertionError( class AttributeError( classmethod( BaseException( compile( BlockingIOError( complex( BrokenPipeError( continue ... Do I need a python3 enabled with readline support, or something? Thanks very much, Laura Creighton From davea at davea.name Tue Feb 10 09:37:57 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 09:37:57 -0500 Subject: Varable parsing error with python In-Reply-To: References: Message-ID: <54DA17C5.6040309@davea.name> On 02/10/2015 03:29 AM, OmPs wrote: > On 10 Feb 2015 13:12, "Chris Angelico" wrote: >> >> On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: >>> def _getPackgeVersion(xmlfile, p): >>> package = str(p) >>> if isinstance(fpmdict["application"]["package"], list): >>> for i in fpmdict["application"]["package"]: >>> if i["@name"] == p: >>> _pkgVersion = i["version"] >>> else: >>> _pkgversion = fpmdict["application"]["package"]["version"] >>> return _pkgVersion >> >> One of your branches doesn't have a return statement in it, so Python >> just returns None. You may want to unindent that return statement one >> level. >> > Tried that as well getting the same error. The code above has at least two errors. Fix them both, post a new version, and tell us the error you now get. First error is that there's no return statement for the case where the first if is true. You might fix that by unindenting, or by adding another return somewhere, or whatever. Second error, depending on how you fixed the first one, is that for some paths through the function, _pkgVersion isn't initialized. So if those conditions happen, and if you fixed the first error, you'll get an exception like: UnboundLocalError: local variable '_pkgVersion' referenced before assignment There also may be logic errors. For example, if there are multiple places where the if i["@name... logic triggers, do you want the _pkgversion to be the first such, or the last such. What do you want for a value if there no matches? -- DaveA From p.f.moore at gmail.com Tue Feb 10 09:40:10 2015 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Feb 2015 06:40:10 -0800 (PST) Subject: Monte Carlo probability calculation in Python In-Reply-To: References: <9a4636dd-5e9a-4c24-ae1c-ac5b447b3039@googlegroups.com> <11497cc2-97e0-4af0-9adb-a8f9955ae774@googlegroups.com> <51d12f89-7d03-41e9-a2c9-3960a15ed6aa@googlegroups.com> <54d55314$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: <9fe18be0-c7be-4532-a3db-1db0a998057d@googlegroups.com> On Monday, 9 February 2015 21:57:51 UTC, Paul Moore wrote: > On Friday, 6 February 2015 23:49:51 UTC, Steven D'Aprano wrote: > > Very nice! Care to share the code? > > Will do. Here's the code I used for the Monopoly calculations. import numpy as np def monopoly(samples): # 2d6 x 3 num = 2 sides = 6 rolls = 3 dice = np.random.randint(1, sides + 1, size=(samples, rolls, num)) # Doubles are if the two dice are the same doubles = dice[...,0] == dice[...,1] # Reroll if this (and all previous rolls) are doubles reroll = np.logical_and.accumulate(doubles, axis=1) # Keep the first entry, and any valid rerolls keep = np.insert(reroll, 0, True, axis=1) # The last column (all 3 are doubles) is "go to gaol" keep, gaol = np.split(keep, [-1], axis=-1) gaol = gaol[...,0] # Add up all the rolls and zero out any we don't want to keep totals = dice.sum(axis=-1) * keep # Remove any "go to gaol" cases totals = totals[~gaol,...] # Add up the distance moved totals = totals.sum(axis=-1) gaol_count = np.sum(gaol) return totals, gaol_count samples = 1000000 totals, gaol_count = monopoly(samples) print("Average distance moved =", totals.mean()) print("Percentage of times you go to gaol: {:%}".format(gaol_count/samples)) for i, count in enumerate(np.bincount(totals)): print("{:>2} {}".format(i, count)) The actual calculations are fairly simple, once you get your head round the tricks you need to do everything array-based. The part I find the hardest by far, is keeping track of the dimensions of the arrays involved, and what I need to do to match up axes, etc. For example, it took me a long time to work out what was going wrong with stripping out the "go to gaol" cases. The problem turned out to be that I needed the line "gaol = gaol[...,0]" to strip off a dimension - but it was really hard working out what was going on, particularly as numpy broadcasts the values you have, so you don't get errors, just weirdly wrong answers. (This isn't unique to numpy, I remember having the same issues when I used to work with J). One thing it'd be nice to do would be to abstract out the array dimension for the number of samples, so that the "application" code could be written as if working on just one sample, and the driver code extended that to multiple samples. But I don't have the first idea how I'd do that without ending up looping in Python, which is what I need to avoid if I want performance to remain good. Lots of scope for more things to learn here, then :-) Thanks again to everyone that helped. Paul From wxjmfauth at gmail.com Tue Feb 10 09:40:22 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 10 Feb 2015 06:40:22 -0800 (PST) Subject: [RELEASE] Python 3.5.0a1 is now available In-Reply-To: References: Message-ID: <887f407e-d88e-421b-943d-b0e433aa76fc@googlegroups.com> Le dimanche 8 f?vrier 2015 23:03:27 UTC+1, Larry Hastings a ?crit?: > On behalf of the > Python development community and the Python 3.5 release > team, I'm also pleased to announce the availability of > Python 3.5.0a1.?? Python 3.5.0a1 is the first alpha release > of Python 3.5, which will be the next major release of > Python.? Python 3.5 is still under heavy development, and is > far from complete. > > > > This is a preview release, and its use is not recommended > for production settings. > > > > > You can download it here: > > https://www.python.org/download/releases/3.5.0a1 > > > > > Happy hacking, > > > > > > /arry It expectedly just does not work. (Not necessary to explain why). jmf From vincent.vande.vyvre at telenet.be Tue Feb 10 10:50:54 2015 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Tue, 10 Feb 2015 16:50:54 +0100 Subject: tabs and the Python3 console In-Reply-To: <201502101436.t1AEafIH032349@fido.openend.se> References: <201502101436.t1AEafIH032349@fido.openend.se> Message-ID: <54DA28DE.9030709@telenet.be> Le 10/02/2015 15:36, Laura Creighton a ?crit : > I have the debian version of python3 installed here. > > Python 3.4.2 (default, Nov 13 2014, 07:01:52) > [GCC 4.9.2] on linux > Type "help", "copyright", "credits" or "license" for more information. > But I cannot seem to type a tab here: > >>>> def fn(**kw): > ... > > (I type a tab here, and get a beep. If I type a tab again I get: > > Display all 178 possibilities? (y or n) > ArithmeticError( chr( > AssertionError( class > AttributeError( classmethod( > BaseException( compile( > BlockingIOError( complex( > BrokenPipeError( continue > > ... > > Do I need a python3 enabled with readline support, or something? > > Thanks very much, > Laura Creighton > > It's a recent change (3.4), use the argument -S if you want the old behaviour https://docs.python.org/3/library/site.html#rlcompleter-config Vincent From davea at davea.name Tue Feb 10 10:51:27 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 10:51:27 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: <54DA1573.1030804@davea.name> Message-ID: <54DA28FF.5010001@davea.name> On 02/10/2015 09:33 AM, Chris Angelico wrote: > On Wed, Feb 11, 2015 at 1:28 AM, Dave Angel wrote: >>> If you can show me a one tuple anywhere in the original code I'll >>> happily buy you a tipple of your choice. >> >> >> print Menu[fav,RandomNum] >> >> was in the original code > > That's not a one-tuple (as in, a tuple with one element), it's a > two-element tuple. The point is that ("3") doesn't create a tuple, > it's just superfluous parentheses. You're right of course. I didn't notice the meaning of one-tuple; I took Mark's comment as if he had said: If you can show me a one tuple anywhere ... The original error message said there was a tuple; I knew that fav wasn't the tuple, but the combination of fav and RandomNum was. -- DaveA From wxjmfauth at gmail.com Tue Feb 10 11:32:27 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 10 Feb 2015 08:32:27 -0800 (PST) Subject: tabs and the Python3 console In-Reply-To: References: Message-ID: <80e218c2-ac8f-455c-8506-445d4d603c43@googlegroups.com> Le mardi 10 f?vrier 2015 15:37:18 UTC+1, Laura Creighton a ?crit?: > I have the debian version of python3 installed here. > > Python 3.4.2 (default, Nov 13 2014, 07:01:52) > [GCC 4.9.2] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> > > But I cannot seem to type a tab here: > > >>> def fn(**kw): > ... > > (I type a tab here, and get a beep. If I type a tab again I get: > > Display all 178 possibilities? (y or n) > ArithmeticError( chr( > AssertionError( class > AttributeError( classmethod( > BaseException( compile( > BlockingIOError( complex( > BrokenPipeError( continue > > ... > > Do I need a python3 enabled with readline support, or something? > > Thanks very much, > Laura Creighton Does it mean, that you can not insert two consecutive tabs because some brillant devs have decided that inserting two consecutive tabs is doing something else than really inserting two tabs? What a world! From lac at openend.se Tue Feb 10 11:41:27 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 10 Feb 2015 17:41:27 +0100 Subject: tabs and the Python3 console In-Reply-To: Message from Vincent Vande Vyvre of "Tue, 10 Feb 2015 16:50:54 +0100." <54DA28DE.9030709@telenet.be> References: <201502101436.t1AEafIH032349@fido.openend.se><54DA28DE.9030709@telenet.be> Message-ID: <201502101641.t1AGfRPW025742@fido.openend.se> In a message of Tue, 10 Feb 2015 16:50:54 +0100, Vincent Vande Vyvre writes: >Le 10/02/2015 15:36, Laura Creighton a ?crit : >> I have the debian version of python3 installed here. >> >> Python 3.4.2 (default, Nov 13 2014, 07:01:52) >> [GCC 4.9.2] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> But I cannot seem to type a tab here: >> >>>>> def fn(**kw): >> ... >> >> (I type a tab here, and get a beep. If I type a tab again I get: >> >> Display all 178 possibilities? (y or n) >> ArithmeticError( chr( >> AssertionError( class >> AttributeError( classmethod( >> BaseException( compile( >> BlockingIOError( complex( >> BrokenPipeError( continue >> >> ... >> >> Do I need a python3 enabled with readline support, or something? >> >> Thanks very much, >> Laura Creighton >> >> > >It's a recent change (3.4), use the argument -S if you want the old >behaviour > >https://docs.python.org/3/library/site.html#rlcompleter-config > >Vincent Thank you very, very much. Laura From ethan at stoneleaf.us Tue Feb 10 11:44:39 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 10 Feb 2015 08:44:39 -0800 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> Message-ID: <54DA3577.6080201@stoneleaf.us> On 02/09/2015 08:46 PM, Chris Angelico wrote: > > class Grid: > blah blah > > def __iter__(self): > for row in range(self._rows): > for col in range(self._cols): > if self._grid[row][col]: > yield self._grid[row][col] I strongly suggest you remove the if self._grid[row][col]: line. Best case scenario: the entire grid is blank, and iterating through it does nothing. Worst case scenario: only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e. with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ian.g.kelly at gmail.com Tue Feb 10 11:53:57 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 10 Feb 2015 09:53:57 -0700 Subject: __next__ and StopIteration In-Reply-To: <54DA3577.6080201@stoneleaf.us> References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> <54DA3577.6080201@stoneleaf.us> Message-ID: On Tue, Feb 10, 2015 at 9:44 AM, Ethan Furman wrote: > On 02/09/2015 08:46 PM, Chris Angelico wrote: >> >> class Grid: >> blah blah >> >> def __iter__(self): >> for row in range(self._rows): >> for col in range(self._cols): >> if self._grid[row][col]: >> yield self._grid[row][col] > > I strongly suggest you remove the > > if self._grid[row][col]: > > line. > > Best case scenario: the entire grid is blank, and iterating through it does nothing. > > Worst case scenario: only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e. > with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected. Depends on what the expected behavior is -- is every grid position something that should be included in the iteration, or are we looking at elements of a container where some possible locations may be empty? You don't expect a dict iteration to yield empty buckets, for example. I have some code that looks similar to this, which is an iterator for a chess board that yields the contained pieces. It doesn't really make sense in that case to yield empty squares. From ckaynor at zindagigames.com Tue Feb 10 12:27:32 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Tue, 10 Feb 2015 09:27:32 -0800 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54d9540d$0$13003$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 10, 2015 at 12:30 AM, Ian Kelly wrote: > > On Mon, Feb 9, 2015 at 5:59 PM, Chris Kaynor wrote: > > On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano > > wrote: > >> so that's an excellent sign that doing so is best practice, but it should > >> not be seen as *required*. After all, perhaps you have good reason for > >> wanting your iterable class to only be iterated over once. > > > > In fact, there is one in the stdlib, the "file" object, which has a > > __iter__ which returns self. The code below shows this, > > Fair point. I suppose that's because the file paradigm itself has a > concept of current position that can't easily be abstracted away. That is basically my thought as well. It would be possible to seek around the file while iterating (seek to the current iteration position, read, then seek back to the previous location), however that would slow down iteration for little practical gain. It would also require locking and possibly introduce odd corner cases. > > > The "file" object is also an example of this. It is technically a > > broken iterator according to the docs: > > > > Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 > > 32 bit (Intel)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> f = open("d:/test.txt") > >>>> iter(f) is f > > True > >>>> for l in f: > > ... print(l) > > ... > > line 1 > > > > line 2 > > > > line 3 > > > >>>> for l in f: > > ... print(l) > > ... > >>>> f.seek(0) > > 0 > >>>> for l in f: > > ... print(l) > > ... > > line 1 > > > > line 2 > > > > line 3 > > > >>>> next(f) > > Traceback (most recent call last): > > File "", line 1, in > > StopIteration > >>>> f.seek(0) > > 0 > >>>> next(f) # This should throw StopIteration as it has previously thrown StopIteration. > > 'line 1\n' > > IIRC the docs also warn that mixing file iteration with other file > methods results in undefined or broken behavior. Maybe that's only for > Python 2 files, however. >From the 2.7.9 docs (https://docs.python.org/2/library/stdtypes.html#file.next): > As a consequence of using a read-ahead buffer, combining next() with other file methods (like readline()) does not work right. However, using seek() to reposition the file to an absolute position will flush the read-ahead buffer. So it explicitly states that seek, the method I used, should work fine. I could not find any similar text in the 3.4 docs. Chris From solarteclark at gmail.com Tue Feb 10 12:27:57 2015 From: solarteclark at gmail.com (solarteclark at gmail.com) Date: Tue, 10 Feb 2015 09:27:57 -0800 (PST) Subject: drawing with the mouse with turtle In-Reply-To: <4cdde466$0$29992$c3e8da3$5496439d@news.astraweb.com> References: <4cdde466$0$29992$c3e8da3$5496439d@news.astraweb.com> Message-ID: excuse me :) i know this somewhat disscussion was still on the past few years way back 2010 can i still ask about something? :) what syntax or coding will i use if i wanted to do multiple strokes of the mouse? because the [turtle.ondrag(turtle.goto)] that you have suggested have really helped but what if i want to write more strokes on different left clicks? like i want to draw 2 lines, like some syntax or code that enables me to add more or just reset the other line and make more or replace it? can you help me :) please :D i know i'm just some random stranger but please help me :D From ethan at stoneleaf.us Tue Feb 10 12:33:23 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 10 Feb 2015 09:33:23 -0800 Subject: __next__ and StopIteration In-Reply-To: References: <54d94307$0$12998$c3e8da3$5496439d@news.astraweb.com> <54D98A35.9060203@earthlink.net> <54DA3577.6080201@stoneleaf.us> Message-ID: <54DA40E3.7030806@stoneleaf.us> On 02/10/2015 08:53 AM, Ian Kelly wrote: > On Tue, Feb 10, 2015 at 9:44 AM, Ethan Furman wrote: >> On 02/09/2015 08:46 PM, Chris Angelico wrote: >>> >>> class Grid: >>> blah blah >>> >>> def __iter__(self): >>> for row in range(self._rows): >>> for col in range(self._cols): >>> if self._grid[row][col]: >>> yield self._grid[row][col] >> >> I strongly suggest you remove the >> >> if self._grid[row][col]: >> >> line. >> >> Best case scenario: the entire grid is blank, and iterating through it does nothing. >> >> Worst case scenario: only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e. >> with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected. > > Depends on what the expected behavior is -- is every grid position > something that should be included in the iteration, or are we looking > at elements of a container where some possible locations may be empty? > You don't expect a dict iteration to yield empty buckets, for example. > > I have some code that looks similar to this, which is an iterator for > a chess board that yields the contained pieces. It doesn't really make > sense in that case to yield empty squares. Cool, thanks for the correction. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From denismfmcmahon at gmail.com Tue Feb 10 13:24:38 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Tue, 10 Feb 2015 18:24:38 +0000 (UTC) Subject: Varable parsing error with python References: Message-ID: On Tue, 10 Feb 2015 18:39:42 +1100, Chris Angelico wrote: > On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: >> def _getPackgeVersion(xmlfile, p): >> package = str(p) >> if isinstance(fpmdict["application"]["package"], list): >> for i in fpmdict["application"]["package"]: >> if i["@name"] == p: >> _pkgVersion = i["version"] >> else: >> _pkgversion = fpmdict["application"]["package"]["version"] >> return _pkgVersion > > One of your branches doesn't have a return statement in it, so Python > just returns None. You may want to unindent that return statement one > level. Even if he unindents the return, it will still return None if it doesn't find an i such that i["@name"] == p, as _pkgVersion only gets set if such a match is found. Could this be cause by eg a string case match issue? Perhaps: if i["@name"].lower() == p.lower(): would be a better comparison to use? -- Denis McMahon, denismfmcmahon at gmail.com From rosuav at gmail.com Tue Feb 10 13:48:37 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 05:48:37 +1100 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: <54DA28FF.5010001@davea.name> References: <54DA1573.1030804@davea.name> <54DA28FF.5010001@davea.name> Message-ID: On Wed, Feb 11, 2015 at 2:51 AM, Dave Angel wrote: > You're right of course. I didn't notice the meaning of one-tuple; I took > Mark's comment as if he had said: > > If you can show me a one tuple anywhere ... Ah, yeah. I see the ambiguity. This is the downside of being so fluent in Typo - sometimes you run into false cognates :) ChrisA From john_ladasky at sbcglobal.net Tue Feb 10 13:53:02 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Tue, 10 Feb 2015 10:53:02 -0800 (PST) Subject: Taming the verbosity of ipython tracebacks Message-ID: <00eab446-bc76-420d-a3bf-62e858451c7b@googlegroups.com> Hi folks, I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 64-bit. Whenever I execute code from within ipython which triggers an exception -- any exception -- I get a page full of information. The exceptions in my code trigger exceptions within ipython. Every error message looks like this: ================================================================ ERROR: Internal Python error in the inspect module. Below is the traceback from this internal error. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 2821, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in << MY ERRONEOUS CODE, AND THE EXCEPTION IT GENERATED, ARE PRINTED HERE >> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 1713, in showtraceback stb = value._render_traceback_() AttributeError: 'NameError' object has no attribute '_render_traceback_' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/IPython/core/ultratb.py", line 759, in structured_traceback records = _fixed_getinnerframes(etb, context, tb_offset) File "/usr/lib/python3/dist-packages/IPython/core/ultratb.py", line 242, in _fixed_getinnerframes records = fix_frame_records_filenames(inspect.getinnerframes(etb, context)) File "/usr/lib/python3.4/inspect.py", line 1332, in getinnerframes framelist.append((tb.tb_frame,) + getframeinfo(tb, context)) File "/usr/lib/python3.4/inspect.py", line 1292, in getframeinfo filename = getsourcefile(frame) or getfile(frame) File "/usr/lib/python3.4/inspect.py", line 583, in getsourcefile if getattr(getmodule(object, filename), '__loader__', None) is not None: File "/usr/lib/python3.4/inspect.py", line 629, in getmodule os.path.realpath(f)] = module.__name__ AttributeError: 'FakeModule' object has no attribute '__name__' Unfortunately, your original traceback can not be constructed. ================================================================ When I run the same script directly from the Linux shell, I get a tidy traceback which tells me exactly what I want, and nothing more. But then, obviously, I lose the interactive ipython shell's capabilities. I would like to get rid of all that fluff surrounding my ipython error message. I'm not sure what the purpose of the inspect module is, or what FakeModule is. I wonder whether I have failed to install something correctly. Any advice is appreciated! From rosuav at gmail.com Tue Feb 10 14:00:14 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 06:00:14 +1100 Subject: Taming the verbosity of ipython tracebacks In-Reply-To: <00eab446-bc76-420d-a3bf-62e858451c7b@googlegroups.com> References: <00eab446-bc76-420d-a3bf-62e858451c7b@googlegroups.com> Message-ID: On Wed, Feb 11, 2015 at 5:53 AM, John Ladasky wrote: > I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 64-bit. > That's nice recent Python, but I just tried installing ipython using pip3, and got version 2.4.1, and was unable to reproduce your scenario. Is it possible you're having problems due to the version mismatch? Can you upgrade your ipython? ChrisA From wxjmfauth at gmail.com Tue Feb 10 14:09:04 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 10 Feb 2015 11:09:04 -0800 (PST) Subject: [RELEASE] Python 3.5.0a1 is now available In-Reply-To: <887f407e-d88e-421b-943d-b0e433aa76fc@googlegroups.com> References: <887f407e-d88e-421b-943d-b0e433aa76fc@googlegroups.com> Message-ID: <0e99b198-ed9f-4788-802e-8c87e5e034b9@googlegroups.com> Le mardi 10 f?vrier 2015 15:40:34 UTC+1, wxjm... at gmail.com a ?crit?: > Le dimanche 8 f?vrier 2015 23:03:27 UTC+1, Larry Hastings a ?crit?: > > On behalf of the > > Python development community and the Python 3.5 release > > team, I'm also pleased to announce the availability of > > Python 3.5.0a1.?? Python 3.5.0a1 is the first alpha release > > of Python 3.5, which will be the next major release of > > Python.? Python 3.5 is still under heavy development, and is > > far from complete. > > > > > > > > This is a preview release, and its use is not recommended > > for production settings. > > > > > > > > > > You can download it here: > > > > https://www.python.org/download/releases/3.5.0a1 > > > > > > > > > > Happy hacking, > > > > > > > > > > > > /arry > > > It expectedly just does not work. > (Not necessary to explain why). > > jmf Also true for IDLE. (End of quick tour.) From fomcl at yahoo.com Tue Feb 10 14:27:08 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Tue, 10 Feb 2015 19:27:08 +0000 (UTC) Subject: Taming the verbosity of ipython tracebacks In-Reply-To: References: Message-ID: <1836362686.666393.1423596428927.JavaMail.yahoo@mail.yahoo.com> ----- Original Message ----- > From: Chris Angelico > To: > Cc: "python-list at python.org" > Sent: Tuesday, February 10, 2015 8:00 PM > Subject: Re: Taming the verbosity of ipython tracebacks > > On Wed, Feb 11, 2015 at 5:53 AM, John Ladasky > wrote: >> I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 > 64-bit. >> > > That's nice recent Python, but I just tried installing ipython using > pip3, and got version 2.4.1, and was unable to reproduce your > scenario. Is it possible you're having problems due to the version > mismatch? Can you upgrade your ipython? Huh? What's pip3? I thought pip works on Python 2 and 3? https://pypi.python.org/pypi/pip/. I always install everything with the pip of my system python, e.g. for Python 3.4 I would do sudo python3.4 $(which pip) install somepackage. Are there any benefits of having multiple pips (except in virtualenvs)? Regards, Albert-Jan From john_ladasky at sbcglobal.net Tue Feb 10 14:29:28 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Tue, 10 Feb 2015 11:29:28 -0800 (PST) Subject: Taming the verbosity of ipython tracebacks In-Reply-To: References: <00eab446-bc76-420d-a3bf-62e858451c7b@googlegroups.com> Message-ID: <56864bde-fc2f-43f1-9952-01d1b03c1099@googlegroups.com> On Tuesday, February 10, 2015 at 11:00:32 AM UTC-8, Chris Angelico wrote: > On Wed, Feb 11, 2015 at 5:53 AM, John Ladasky > wrote: > > I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 64-bit. > > > > That's nice recent Python, but I just tried installing ipython using > pip3, and got version 2.4.1, and was unable to reproduce your > scenario. Is it possible you're having problems due to the version > mismatch? Can you upgrade your ipython? > > ChrisA Thank you Chris, that did the trick. I now have ipython 2.4.1 as well, and my tracebacks are behaving. Ubuntu's repository apparently has an outdated version of ipython. Also, a heads-up: the pip3 utility does not recognize a package named ipython3, the name that Ubuntu expects. The package name reverts to plain-old ipython. This name conflict might cause problems for someone who wants to run ipython with Py2 -- while that isn't me, it may affect some of you. From john_ladasky at sbcglobal.net Tue Feb 10 14:43:02 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Tue, 10 Feb 2015 11:43:02 -0800 (PST) Subject: Taming the verbosity of ipython tracebacks In-Reply-To: References: Message-ID: On Tuesday, February 10, 2015 at 11:34:12 AM UTC-8, Albert-Jan Roskam wrote: > ----- Original Message ----- > > > From: Chris Angelico > > To: > > Cc: "python-list at python.org" > > Sent: Tuesday, February 10, 2015 8:00 PM > > Subject: Re: Taming the verbosity of ipython tracebacks > > > > On Wed, Feb 11, 2015 at 5:53 AM, John Ladasky > > wrote: > >> I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 > > 64-bit. > >> > > > > That's nice recent Python, but I just tried installing ipython using > > pip3, and got version 2.4.1, and was unable to reproduce your > > scenario. Is it possible you're having problems due to the version > > mismatch? Can you upgrade your ipython? > > > Huh? What's pip3? I thought pip works on Python 2 and 3? > https://pypi.python.org/pypi/pip/. I always install everything with the pip of my system python, e.g. for Python 3.4 I would do sudo python3.4 $(which pip) install somepackage. Are there any benefits of having multiple pips (except in virtualenvs)? > > Regards, > Albert-Jan Albert-Jan, on my Ubuntu system, pip and pip3 are two different utilities. Anything that I install with pip is linked to my Python 2 interpreter; likewise, pip3 links to Python 3. From davea at davea.name Tue Feb 10 15:38:02 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 15:38:02 -0500 Subject: TypeError: list indices must be integers, not tuple In-Reply-To: References: <54DA1573.1030804@davea.name> <54DA28FF.5010001@davea.name> Message-ID: <54DA6C2A.1030805@davea.name> On 02/10/2015 01:48 PM, Chris Angelico wrote: > On Wed, Feb 11, 2015 at 2:51 AM, Dave Angel wrote: >> You're right of course. I didn't notice the meaning of one-tuple; I took >> Mark's comment as if he had said: >> >> If you can show me a one tuple anywhere ... > > Ah, yeah. I see the ambiguity. This is the downside of being so fluent > in Typo - sometimes you run into false cognates :) > > ChrisA > If somebody were still doing py-quote-of-the-week, I'd nominate that sentence: This is the downside of being so fluent in Typo - sometimes you run into false cognates :) -- DaveA From davea at davea.name Tue Feb 10 15:40:45 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 15:40:45 -0500 Subject: Varable parsing error with python In-Reply-To: References: Message-ID: <54DA6CCD.803@davea.name> On 02/10/2015 01:24 PM, Denis McMahon wrote: > On Tue, 10 Feb 2015 18:39:42 +1100, Chris Angelico wrote: > >> On Tue, Feb 10, 2015 at 6:30 PM, OmPs wrote: >>> def _getPackgeVersion(xmlfile, p): >>> package = str(p) >>> if isinstance(fpmdict["application"]["package"], list): >>> for i in fpmdict["application"]["package"]: >>> if i["@name"] == p: >>> _pkgVersion = i["version"] >>> else: >>> _pkgversion = fpmdict["application"]["package"]["version"] >>> return _pkgVersion >> >> One of your branches doesn't have a return statement in it, so Python >> just returns None. You may want to unindent that return statement one >> level. > > Even if he unindents the return, it will still return None if it doesn't > find an i such that i["@name"] == p, as _pkgVersion only gets set if such > a match is found. But if _pkgVersion weren't set, he'd be getting UnboundLocalError exception, not a None value. > -- DaveA From breamoreboy at yahoo.co.uk Tue Feb 10 15:47:17 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 10 Feb 2015 20:47:17 +0000 Subject: Taming the verbosity of ipython tracebacks In-Reply-To: <1836362686.666393.1423596428927.JavaMail.yahoo@mail.yahoo.com> References: <1836362686.666393.1423596428927.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 10/02/2015 19:27, Albert-Jan Roskam wrote: > > > ----- Original Message ----- > >> From: Chris Angelico >> To: >> Cc: "python-list at python.org" >> Sent: Tuesday, February 10, 2015 8:00 PM >> Subject: Re: Taming the verbosity of ipython tracebacks >> >> On Wed, Feb 11, 2015 at 5:53 AM, John Ladasky >> wrote: >>> I'm running Python 3.4.0, and ipython3 1.2.1, on Ubuntu Linux 14.04 >> 64-bit. >>> >> >> That's nice recent Python, but I just tried installing ipython using >> pip3, and got version 2.4.1, and was unable to reproduce your >> scenario. Is it possible you're having problems due to the version >> mismatch? Can you upgrade your ipython? > > > Huh? What's pip3? I thought pip works on Python 2 and 3? > https://pypi.python.org/pypi/pip/. I always install everything with the pip of my system python, e.g. for Python 3.4 I would do sudo python3.4 $(which pip) install somepackage. Are there any benefits of having multiple pips (except in virtualenvs)? > > Regards, > Albert-Jan > On Windows under C:\Python34\Scripts I have pip.exe, pip3.exe and pip3.4.exe. I believe that they are identical. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From skip.montanaro at gmail.com Tue Feb 10 16:05:47 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 10 Feb 2015 15:05:47 -0600 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? Message-ID: I know this is way off-topic for this group, but I figured if anyone in the online virtual communities I participate in would know the answer, the Pythonistas would... Google has so far not been my friend in this realm. One of the things I really like about my Skype keyboard (and likely other "soft" keyboards on Android) is that when you hold down a "key" for a brief moment, a little mini keyboard pops up, from which you can easily choose various accented variants and other symbols. For instance, If I press and hold the "d" key, I see these choices (ignore the capitalization of the first letter - my mistake sending a text message to myself from my phone, and I can't seem to convert it to lower case): ?|?&d?? While I'm a touch typist, I almost never use auto-repeat, which is the "binding" of held keys in most environments (curse you, IBM and your Selectric!). These days I find my self needing accented characters much more frequently than key repeat (C-u 2 5 - suffices in Emacs to bat out 25 hyphens). Being an American with an American keyboard, I haven't the slightest idea how to type any accented characters or common symbols using the many modifier keys on my keyboard, and no key caps display what the various options are. And I'm getting kind of tired of going to Google and searching for "degree symbol". :-/ Is there an X11 or Mac extension/program/app/magic thing which I can install in either environment to get this kind of functionality? I'm thinking that if you hold down a key for the auto-repeat interval, instead of the key repeat thing making all sorts of duplicates, a little window would pop up over/near the insertion point, which I can navigate with the arrow keys, then hit RET to accept or ESC (or similar) to cancel. It need not be perfect. It might (for example) only work in certain environments (Chrome, Emacs, vim, Firefox). Anyplace to start. It need even be written in Python (though that would be cool.) I think that once something like this caught hold, it would fairly quickly take over from the dark lords of auto-repeat. Thx, Skip From ethan at stoneleaf.us Tue Feb 10 16:18:45 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 10 Feb 2015 13:18:45 -0800 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <54DA75B5.4090005@stoneleaf.us> On 02/10/2015 01:05 PM, Skip Montanaro wrote: > [request for alternate function when key is held down] Not exactly what you asked for, but here's how to combine characters in Vim: http://vim.wikia.com/wiki/Entering_special_characters Hopefully somebody knows about a pop-up window type thingie [1]. -- ~Ethan~ [1] Yes, that's the technical name for it ;) okay, okay, not really. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From python.list at tim.thechases.com Tue Feb 10 16:29:00 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 10 Feb 2015 15:29:00 -0600 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <20150210152900.54129700@bigbox.christie.dr> On 2015-02-10 15:05, Skip Montanaro wrote: > For instance, If I press and hold the "d" key, I see these choices > (ignore the capitalization of the first letter - my mistake sending > a text message to myself from my phone, and I can't seem to convert > it to lower case): ?|?&d?? > > I haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard > > Is there an X11 or Mac extension/program/app/magic thing which I can > install in either environment to get this kind of functionality? While it's not exactly a hold-down-get-a-menu, I opt for changing my (otherwise-useless) caps-lock key to an X compose key: $ setxkbmap -option compose:caps I can then hit caps-lock followed by what are generally intuitive sequences. For your first one, that would be "capital-D minus". I'm not sure what the other characters are supposed to be, so I'm not sure how to find them. But ? is "compose, e, apostrophe", ? is "compose, n, tilde", the degree sign is "compose, o, o", the ? is "compose, E, equals", etc. There are loads of these documented in (on my machine, where my locale is en_US.UTF-8) /usr/share/X11/locale/en_US.UTF-8/Compose Some of them are a little less intuitive, though the majority of the time I can just guess them (I'd never typed "?" before, but guessed and was right). Otherwise I search that above file. This also has the advantage that it should work in every X application, including Unicode-aware terminal applications (in Unicode-aware terminals). Adding some sort of press-and-hold UI would limit it to those applications that chose to support it (or even *could* support it). > While I'm a touch typist, I almost never use auto-repeat, which is > the "binding" of held keys in most environments I agree, as vi/vim makes it easy to insert multiples of the same character (or characters) akin to what you describe in Emacs. -tkc From lac at openend.se Tue Feb 10 16:33:26 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 10 Feb 2015 22:33:26 +0100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: Message from Skip Montanaro of "Tue, 10 Feb 2015 15:05:47 -0600." References: Message-ID: <201502102133.t1ALXQ4L021850@fido.openend.se> I'm using this: http://michel.staelens.pagesperso-orange.fr/clavier/index_GB.htm# to get cyrillic. Not sure the other alternatives will get you what you want -- my keyboard is rather well loaded with accented letters from the get-go. Laura From rosuav at gmail.com Tue Feb 10 17:12:43 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 09:12:43 +1100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <201502102133.t1ALXQ4L021850@fido.openend.se> References: <201502102133.t1ALXQ4L021850@fido.openend.se> Message-ID: On Wed, Feb 11, 2015 at 8:33 AM, Laura Creighton wrote: > I'm using this: > http://michel.staelens.pagesperso-orange.fr/clavier/index_GB.htm# > to get cyrillic. Not sure the other alternatives will get you what > you want -- my keyboard is rather well loaded with accented letters > from the get-go. That looks a lot more general than what I use: https://github.com/Rosuav/shed/blob/master/translit.pike The code's a bit messy, largely because the human languages involved are complex, but broadly, here's the plan: Mode 1: Reversible transliteration. In any of several modes (Cyrillic (Russian, Ukrainian, and Serbian variants), Korean, Elder Futhark, and Greek), it converts the input ASCII text into a non-ASCII script, and back again. Normally I use that to key in text that I can't otherwise type, but I can also copy and paste the other script and check it. Some of them also support diacriticals, as per the other mode. Mode 2: Latin-only mode. Fairly simple: the 'diacriticals' function just transforms some backslash escapes into individual Unicode characters, usually the combining ones, and then performs an NFC normalization on the resulting string. So, for instance (using Python string literal format; in actual usage, these would be single backslashes): Input: "\\!Sue\\'ltalo!" Step 1: "\u00A1Sue\u0301ltalo!" Step 2:"\u00A1Sue\u0301ltalo!" Step 3: "\u00A1Su\u00E9ltalo!" Displayed text: ?Su?ltalo! In usage, it's pretty simple. You key in a letter, then you hit backslash and a (usually mnemonically-appropriate) symbol, and it combines in an appropriate diacritical mark. There are a few oddities in the transformation table (for instance, "i\." produces a dotless i, but "I\." produces a dotted I; and "\-" produces a combining macron, unless it follows a D or d, in which case it produces a crossed D), so if you want to adapt this for your own use, you may want to throw out some of the transformations. But if you only occasionally have need of these kinds of letters, it may be easiest to just keep this handy, key in what you want, and then clipboard it to whatever you want to work with. ChrisA From davea at davea.name Tue Feb 10 17:14:50 2015 From: davea at davea.name (Dave Angel) Date: Tue, 10 Feb 2015 17:14:50 -0500 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <54DA82DA.2060506@davea.name> On 02/10/2015 04:05 PM, Skip Montanaro wrote: > I know this is way off-topic for this group, but I figured if anyone > in the online virtual communities I participate in would know the > answer, the Pythonistas would... Google has so far not been my friend > in this realm. > > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. For > instance, If I press and hold the "d" key, I see these choices (ignore > the capitalization of the first letter - my mistake sending a text > message to myself from my phone, and I can't seem to convert it to > lower case): ?|?&d?? > > While I'm a touch typist, I almost never use auto-repeat, which is the > "binding" of held keys in most environments (curse you, IBM and your > Selectric!). These days I find my self needing accented characters > much more frequently than key repeat (C-u 2 5 - suffices in Emacs to > bat out 25 hyphens). Being an American with an American keyboard, I > haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard, and no key > caps display what the various options are. And I'm getting kind of > tired of going to Google and searching for "degree symbol". :-/ > > Is there an X11 or Mac extension/program/app/magic thing which I can > install in either environment to get this kind of functionality? I'm > thinking that if you hold down a key for the auto-repeat interval, > instead of the key repeat thing making all sorts of duplicates, a > little window would pop up over/near the insertion point, which I can > navigate with the arrow keys, then hit RET to accept or ESC (or > similar) to cancel. It need not be perfect. It might (for example) > only work in certain environments (Chrome, Emacs, vim, Firefox). > Anyplace to start. It need even be written in Python (though that > would be cool.) I think that once something like this caught hold, it > would fairly quickly take over from the dark lords of auto-repeat. > > Thx, > > Skip > On Ubuntu 14.04, you can get a character map from the bar at the top. In my case, the icon is labeled "En" standing for English. I thought it was pretty dumb, till I realized if you click on it, then choose "Character", it gives you a selectable set of characters, which can then be copied to clipboard, etc. It's a mouse thing, so it's not what you ask, but it is a start. -- DaveA From lac at openend.se Tue Feb 10 17:15:11 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 10 Feb 2015 23:15:11 +0100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: Message from Tim Chase of "Tue, 10 Feb 2015 15:29:00 -0600." <20150210152900.54129700@bigbox.christie.dr> References: <20150210152900.54129700@bigbox.christie.dr> Message-ID: <201502102215.t1AMFBAB031123@fido.openend.se> In a message of Tue, 10 Feb 2015 15:29:00 -0600, Tim Chase writes: >While it's not exactly a hold-down-get-a-menu, I opt for changing my >(otherwise-useless) caps-lock key to an X compose key: > > $ setxkbmap -option compose:caps > >I can then hit caps-lock followed by what are generally intuitive >sequences. For your first one, that would be "capital-D minus". I'm >not sure what the other characters are supposed to be, so I'm not >sure how to find them. But ? is "compose, e, apostrophe", ? is >"compose, n, tilde", the degree sign is "compose, o, o", the ? is >"compose, E, equals", etc. There are loads of these documented in (on >my machine, where my locale is en_US.UTF-8) >/usr/share/X11/locale/en_US.UTF-8/Compose > >Some of them are a little less intuitive, though the majority of the >time I can just guess them (I'd never typed "?" before, but guessed >and was right). Otherwise I search that above file. > >This also has the advantage that it should work in every X >application, including Unicode-aware terminal applications (in >Unicode-aware terminals). Adding some sort of press-and-hold UI >would limit it to those applications that chose to support it (or >even *could* support it). > >> While I'm a touch typist, I almost never use auto-repeat, which is >> the "binding" of held keys in most environments > >I agree, as vi/vim makes it easy to insert multiples of the same >character (or characters) akin to what you describe in Emacs. > >-tkc Wow. US keyboards do not come with a 'compose' key, then? It just never occurred to me that Skip might be missing one. Oh, goodness gracious then, go with this solution. Much better than mine --though the one I pointed at is great should you suddenly need to type something in cyrillic while at a non-cyrillic keyboard. Laura From invalid at invalid.invalid Tue Feb 10 18:15:16 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 10 Feb 2015 23:15:16 +0000 (UTC) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <20150210152900.54129700@bigbox.christie.dr> Message-ID: On 2015-02-10, Laura Creighton wrote: > Wow. US keyboards do not come with a 'compose' key, then? Nope. > It just never occurred to me that Skip might be missing one. I always configure one of my "extra" keys (windows-key, menu-key, right-ctrl, etc.) as compose. But, I use it infrequenty enough that it's probably rather amusing watching me try to find it via trial and error. -- Grant Edwards grant.b.edwards Yow! I own seven-eighths of at all the artists in downtown gmail.com Burbank! From steve+comp.lang.python at pearwood.info Tue Feb 10 18:25:12 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 10:25:12 +1100 Subject: tabs and the Python3 console References: Message-ID: <54da9359$0$13005$c3e8da3$5496439d@news.astraweb.com> Laura Creighton wrote: > I have the debian version of python3 installed here. > > Python 3.4.2 (default, Nov 13 2014, 07:01:52) > [GCC 4.9.2] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> > > But I cannot seem to type a tab here: > >>>> def fn(**kw): > ... > > (I type a tab here, and get a beep. If I type a tab again I get: > > Display all 178 possibilities? (y or n) > ArithmeticError( chr( > AssertionError( class > AttributeError( classmethod( > BaseException( compile( > BlockingIOError( complex( > BrokenPipeError( continue > > ... > > Do I need a python3 enabled with readline support, or something? The good news is that after many, many years of supporting tab completion, Python 3.4 finally enables it by default. The bad news is that its handling of the tab key is fundamentally broken. There are a couple of tasks on the bug tracker dealing with this. I'm not able to search for them right now, but you may be able to find them: there is a task to enable tab completion by default, which I expect is closed by now, and (if memory serves me correctly) a second, open, task to fix the tab-for-indentation issue. As an alternative, you could try my tab completion and history module: http://code.google.com/p/tabhistory/ I've been using it on Linux for about three or four years, and although I don't promise it is bug-free, it shouldn't blow up your computer :-) You can read an announcement here: http://code.activestate.com/lists/python-list/672898/ Feedback from Mac and Windows users is very, very welcome. -- Steven From vlyamtsev at gmail.com Tue Feb 10 18:38:02 2015 From: vlyamtsev at gmail.com (vlyamtsev at gmail.com) Date: Tue, 10 Feb 2015 15:38:02 -0800 (PST) Subject: function inclusion problem Message-ID: I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: import mydef ... Fatalln "my test" i have NameError: name 'Fatalln' is not defined I also tried include('mydef.py') with the same result... What is the right syntax? Thanks From steve+comp.lang.python at pearwood.info Tue Feb 10 18:55:17 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 10:55:17 +1100 Subject: Varable parsing error with python References: Message-ID: <54da9a68$0$13013$c3e8da3$5496439d@news.astraweb.com> OmPs wrote: > On 10 Feb 2015 13:59, "OmPs" wrote: >> Tried that as well getting the same error. > > I feel its something to do with variable substitution. Python doesn't do variable substitution. At least not the way I think of it. What do you mean by "variable substitution"? Instead of "feeling" what the problem is, let's do some old-fashioned debugging. Start by reading through the code, and in your head (or on paper) trace through the possible lines that may be reached. The _getPackgeVersion function has only 11 lines, it isn't hard. Here is your function again: def _getPackgeVersion(xmlfile, p): package = str(p) if isinstance(fpmdict["application"]["package"], list): for i in fpmdict["application"]["package"]: if i["@name"] == p: _pkgVersion = i["version"] else: _pkgversion = fpmdict["application"]["package"]["version"] return _pkgVersion I can immediately see four problems: (1) One branch of the "if isinstance" fails to return. So if the function takes that branch, it will finish the loop, then exit the if...else clause, then hit the end of the function and return None. (2) Sometimes that branch may fail to set the _pkgVersion variable at all, if none of the "@name" keys equal p. (3) The other branch fails to set the _pkgVersion variable. Python is case-sensitive, so _pkgversion and _pkgVersion are different variables. (4) Your package variable is never used. I don't know how to fix problem #2 since I don't know what behaviour would be useful to you. So I'll just leave that for you. I can fix problems #1 #3 and #4: def _getPackgeVersion(xmlfile, p): if isinstance(fpmdict["application"]["package"], list): for i in fpmdict["application"]["package"]: if i["@name"] == p: _pkgVersion = i["version"] else: _pkgVersion = fpmdict["application"]["package"]["version"] return _pkgVersion Now we can see that if _getPackgeVersion still returns None, the only way it can do so is if the "version" key is set to have None as a value. Are you sure that there is a "version" key set? I can see another problem, earlier in your code: for i in doc.get("application").get("instance", None): ... That does not do what you expect. If there is no "instance" key, the second get will return None, which is not iterable: py> for i in None: ... pass ... Traceback (most recent call last): File "", line 1, in TypeError: 'NoneType' object is not iterable -- Steven From sohcahtoa82 at gmail.com Tue Feb 10 18:55:38 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Tue, 10 Feb 2015 15:55:38 -0800 (PST) Subject: function inclusion problem In-Reply-To: References: Message-ID: <48ec9b8f-8340-490e-8ea4-c4c45c1e43b7@googlegroups.com> On Tuesday, February 10, 2015 at 3:38:12 PM UTC-8, vlya... at gmail.com wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: > import mydef > ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? > Thanks It would help us help you a lot of you copy/paste your code from both mydef.py and test.py so we can see exactly what you're doing. Don't re-type what you entered, because people (Especially new programmers) are prone to either making typos or leaving out certain things because they don't think they're important. Copy/Paste the code from the two files and then copy/paste the error you're getting. From steve+comp.lang.python at pearwood.info Tue Feb 10 18:57:52 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 10:57:52 +1100 Subject: function inclusion problem References: Message-ID: <54da9b01$0$13013$c3e8da3$5496439d@news.astraweb.com> vlyamtsev at gmail.com wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it > from "mydef.py", but when i try to call it from "test.py" in the same > folder: import mydef ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? > Thanks Preferred: import mydef mydef.Fatalln("my test") Also acceptable: from mydef import Fatalln Fatalln("my test") -- Steven From torriem at gmail.com Tue Feb 10 19:00:14 2015 From: torriem at gmail.com (Michael Torrie) Date: Tue, 10 Feb 2015 17:00:14 -0700 Subject: function inclusion problem In-Reply-To: References: Message-ID: <54DA9B8E.3090806@gmail.com> On 02/10/2015 04:38 PM, vlyamtsev at gmail.com wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: > import mydef > ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? Almost. Try this: mydef.Fatalln() Unless you import the symbols from your mydef module into your program they have to referenced by the module name. This is a good thing and it helps keep your code separated and clean. It is possible to import individual symbols from a module like this: from mydef import Fatalln Avoid the temptation to import *all* symbols from a module into the current program's namespace. Better to type out the extra bit. Alternatively you can alias imports like this import somemodule.submodule as foo Frequently this idiom is used when working with numpy to save a bit of time, while preserving the separate namespaces. From ian.g.kelly at gmail.com Tue Feb 10 19:02:35 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 10 Feb 2015 17:02:35 -0700 Subject: function inclusion problem In-Reply-To: References: Message-ID: On Tue, Feb 10, 2015 at 4:38 PM, wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: > import mydef > ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? import mydef mydef.Fatalin("my test") or from mydef import Fatalin Fatalin("my test") From sohcahtoa82 at gmail.com Tue Feb 10 19:02:53 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Tue, 10 Feb 2015 16:02:53 -0800 (PST) Subject: function inclusion problem In-Reply-To: References: Message-ID: <4a453b87-efce-40bc-b512-e51267917703@googlegroups.com> On Tuesday, February 10, 2015 at 3:38:12 PM UTC-8, vlya... at gmail.com wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: > import mydef > ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? > Thanks If you only do `import mydef`, then it creates a module object called `mydef` which contains all the global members in mydef.py. When you want to call a function from that module, you need to specify that you're calling a function from that module by putting the module name followed by a period, then the function. For example: mydef.Fatalln("my test") If you wanted to be able to call Fatalln without using the module name, you could import just the Fatalln function: from mydef import Fatalln Fatalln("my test") If you had a lot of functions in mydef.py and wanted to be able to access them all without that pesky module name, you could also do: from mydef import * However, this can often be considered a bad practice as you're polluting your global name space, though can be acceptable in specific scenarios. For more information, check https://docs.python.org/3/tutorial/modules.html From lac at openend.se Tue Feb 10 19:06:00 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 11 Feb 2015 01:06:00 +0100 Subject: function inclusion problem In-Reply-To: Message from vlyamtsev@gmail.com of "Tue, 10 Feb 2015 15:38:02 -0800." References: Message-ID: <201502110006.t1B060VZ021974@fido.openend.se> In a message of Tue, 10 Feb 2015 15:38:02 -0800, vlyamtsev at gmail.com writes: >I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: >import mydef >... >Fatalln "my test" >i have NameError: name 'Fatalln' is not defined >I also tried include('mydef.py') with the same result... >What is the right syntax? >Thanks >-- >https://mail.python.org/mailman/listinfo/python-list from mydef import Fatalln From ndbecker2 at gmail.com Tue Feb 10 19:06:34 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 10 Feb 2015 19:06:34 -0500 Subject: line_profiler: what am I doing wrong? Message-ID: I inserted @profile def run(...) into a module-level global function called 'run'. Something is very wrong here. 1. profile results were written before anything even ran 2. profile is not defined? kernprof -l ./test_unframed.py --lots --of --args ... Wrote profile results to test_unframed.py.lprof Traceback (most recent call last): File "/home/nbecker/.local/bin/kernprof", line 9, in load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, in main execfile(script_file, ns, ns) File "./test_unframed.py", line 721, in @profile NameError: name 'profile' is not defined From pkpearson at nowhere.invalid Tue Feb 10 19:10:42 2015 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 11 Feb 2015 00:10:42 GMT Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: Message-ID: On Tue, 10 Feb 2015 15:05:47 -0600, Skip Montanaro wrote: [snip] > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. [snip] > (C-u 2 5 - suffices in Emacs to > bat out 25 hyphens). Being an American with an American keyboard, I > haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard, and no key > caps display what the various options are. And I'm getting kind of > tired of going to Google and searching for "degree symbol". :-/ [snip] Again, not what you asked for, but since you use Emacs . . . Are you aware of Emacs's control-backslash ("toggle-input-method") command? If I'm going to be writing French or Spanish, I hit control-backslash and type "latin-prefix". Then, until I hit control-backslash again, "'e" becomes ?, "~n" becomes ?, '"o' becomes ?, and many other not-too-hard-to-remember things. Many other "input methods" are available for other languages. It's actually reminiscent of how Spanish typewriters used to work, with an accent key that didn't advance the carriage. -- To email me, substitute nowhere->runbox, invalid->com. From ethan at stoneleaf.us Tue Feb 10 19:11:35 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 10 Feb 2015 16:11:35 -0800 Subject: line_profiler: what am I doing wrong? In-Reply-To: References: Message-ID: <54DA9E37.9090602@stoneleaf.us> On 02/10/2015 04:06 PM, Neal Becker wrote: > I inserted > @profile > def run(...) > > into a module-level global function called 'run'. Something is very wrong here. > 1. profile results were written before anything even ran > 2. profile is not defined? > > kernprof -l ./test_unframed.py --lots --of --args ... > > Wrote profile results to test_unframed.py.lprof > Traceback (most recent call last): > File "/home/nbecker/.local/bin/kernprof", line 9, in > load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() > File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, > in main > execfile(script_file, ns, ns) > File "./test_unframed.py", line 721, in > @profile > NameError: name 'profile' is not defined I'm going to guess that writing the profile results is in a try/finally -- so first you see the results being written, then the exception that triggered. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From lac at openend.se Tue Feb 10 19:16:31 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 11 Feb 2015 01:16:31 +0100 Subject: function inclusion problem In-Reply-To: Message from Laura Creighton of "Wed, 11 Feb 2015 01:06:00 +0100." <201502110006.t1B060VZ021974@fido.openend.se> References: <201502110006.t1B060VZ021974@fido.openend.se> Message-ID: <201502110016.t1B0GVah024279@fido.openend.se> In a message of Wed, 11 Feb 2015 01:06:00 +0100, Laura Creighton writes: >In a message of Tue, 10 Feb 2015 15:38:02 -0800, vlyamtsev at gmail.com writes: >>I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: >>import mydef >>... >>Fatalln "my test" >>i have NameError: name 'Fatalln' is not defined >>I also tried include('mydef.py') with the same result... >>What is the right syntax? >>Thanks >>-- >>https://mail.python.org/mailman/listinfo/python-list > >from mydef import Fatalln > Also, please be warned. If you use a unix system, or a linux system. There are lots of problems you can get into if you expect something named 'test' to run your code. Because they already have one in their shell, and that one wins, and so ... well, test.py is safe. But if you rename it as a script and call it the binary file test ... Bad and unexpected things happen. Name it 'testme' or something like that. Never have that problem again. :) Been there, done that! Laura From matt at numenta.org Tue Feb 10 19:17:48 2015 From: matt at numenta.org (Matthew Taylor) Date: Tue, 10 Feb 2015 16:17:48 -0800 Subject: building c extensions with setuptools that are not tied to python installed on build machine Message-ID: Hello pythonistas, This is my first message to this mailing list, so if my question is off-topic, please direct me to the correct mailing list. I manage the NuPIC [1] open source machine intelligence project, which is a python project with C extensions. We have a build on Travis-CI [2] that creates binary wheels on release tags (in git). I'm having a problem with OS X binaries. The Travis-CI build machine is OS X 10.9 and uses the default Python 2.7.6 that comes with OS X. When binaries are created on this machine, they seem to be linked somehow to the default Python, because when a user running a homebrew-installed python (or Anaconda) attempts to install the binary, there's an awful problem [3], even when the Python versions are exactly the same (2.7.6). It seems that some connection is made at compile time when C extensions are built that link the binary file to the system Python on the OS X machine building the binary. This means any users working with the homebrew Python or Anaconda will get nasty thread errors when the program attempts to run against two Pythons. Fatal Python error: PyThreadState_Get: no current thread Does this make sense to anyone? I'm still a little new to Python in general (especially binary packaging), and it seems like this would be a common problem for any projects with C extensions that need broad binary distribution. Does anyone have any suggestions? Please take a look at our setup.py file [4] and tell me if we're doing something egregiously wrong. [1] https://github.com/numenta/nupic [2] https://travis-ci.org/numenta/nupic [3] https://github.com/numenta/nupic/issues/1813 [4] https://github.com/numenta/nupic/blob/master/setup.py Thanks thanks in advance, --------- Matt Taylor OS Community Flag-Bearer Numenta From ndbecker2 at gmail.com Tue Feb 10 19:18:56 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 10 Feb 2015 19:18:56 -0500 Subject: line_profiler: what am I doing wrong? References: <54DA9E37.9090602@stoneleaf.us> Message-ID: Ethan Furman wrote: > On 02/10/2015 04:06 PM, Neal Becker wrote: >> I inserted >> @profile >> def run(...) >> >> into a module-level global function called 'run'. Something is very wrong >> here. 1. profile results were written before anything even ran >> 2. profile is not defined? >> >> kernprof -l ./test_unframed.py --lots --of --args ... >> >> Wrote profile results to test_unframed.py.lprof >> Traceback (most recent call last): >> File "/home/nbecker/.local/bin/kernprof", line 9, in >> load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() >> File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line >> 221, >> in main >> execfile(script_file, ns, ns) >> File "./test_unframed.py", line 721, in >> @profile >> NameError: name 'profile' is not defined > > I'm going to guess that writing the profile results is in a try/finally -- so > first you see the results being written, then the exception that triggered. > > -- > ~Ethan~ I believe you are suggesting the apparent out-of-order is due to try/finally, but kernprof is supposed to inject 'profile' into the global namespace, so @profile should be defined - I don't know why it isn't working. -- -- Those who don't understand recursion are doomed to repeat it From steve+comp.lang.python at pearwood.info Tue Feb 10 19:27:01 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 11:27:01 +1100 Subject: line_profiler: what am I doing wrong? References: Message-ID: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> Neal Becker wrote: > I inserted > @profile > def run(...) > > into a module-level global function called 'run'. Something is very wrong > here. 1. profile results were written before anything even ran > 2. profile is not defined? Well, is it defined? Where does it come from? If you defined it yourself, it needs to be defined before you can use it. This won't work: @profile def run(...) def profile(func): ... Swap the order of profile and run and it should work. (Give or take any additional bugs in your code.) If you've imported it from an external module, how did you import it? import some_module @some_module.profile def run(...) should work. So will this: from some_module import profile @profile def run(...) But this won't: import some_module @profile def run(...) and will fail with NameError, exactly as you are experiencing. -- Steven From ndbecker2 at gmail.com Tue Feb 10 19:45:05 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 10 Feb 2015 19:45:05 -0500 Subject: line_profiler: what am I doing wrong? References: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > Neal Becker wrote: > >> I inserted >> @profile >> def run(...) >> >> into a module-level global function called 'run'. Something is very wrong >> here. 1. profile results were written before anything even ran >> 2. profile is not defined? > > Well, is it defined? Where does it come from? > > If you defined it yourself, it needs to be defined before you can use it. > This won't work: > > > @profile > def run(...) > > def profile(func): ... > > > Swap the order of profile and run and it should work. (Give or take any > additional bugs in your code.) > > > If you've imported it from an external module, how did you import it? > > > import some_module > > @some_module.profile > def run(...) > > > should work. So will this: > > > from some_module import profile > > @profile > def run(...) > > > But this won't: > > > import some_module > > @profile > def run(...) > > > and will fail with NameError, exactly as you are experiencing. > > > > To quote from https://pypi.python.org/pypi/line_profiler/ $ kernprof -l script_to_profile.py kernprof will create an instance of LineProfiler and insert it into the __builtins__ namespace with the name profile. It has been written to be used as a decorator, so in your script, you decorate the functions you want to profile with @profile. @profile def slow_function(a, b, c): ... I've used it before (maybe 1 year ago), don't know why it isn't working now. -- -- Those who don't understand recursion are doomed to repeat it From steve+comp.lang.python at pearwood.info Tue Feb 10 20:17:22 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 12:17:22 +1100 Subject: line_profiler: what am I doing wrong? References: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54daada4$0$12998$c3e8da3$5496439d@news.astraweb.com> Neal Becker wrote: > To quote from https://pypi.python.org/pypi/line_profiler/ > > $ kernprof -l script_to_profile.py > kernprof will create an instance of LineProfiler and insert it into the > __builtins__ namespace with the name profile. Ewww!!!! What a Ruby-esque interface, that makes me sad :-( And what if you have your own profile global name? And *wrong* too. `__builtins__` is a private CPython implementation detail. The way to monkey-patch the built-ins in Python 2 is to inject the object into `__builtin__` (no s), or `builtins` in Python 3. Seeing as line_profiler is written in C, perhaps the author (Robert Kern) doesn't care about supporting Jython or IronPython, but there may be Python implementations (PyPy perhaps?) which can run C code but don't have __builtins__. In any case, it sounds like it isn't working. Try creating this simple file: # demo.py print(profile) # Does profile actually get injected into the builtins? then running it: kernprof -l demo.py If that still gives a NameError, you have a bug in kernprof. Or possibly you have broken it -- perhaps you are accidentally shadowing one of its modules with a module of your own? -- Steven From rustompmody at gmail.com Tue Feb 10 22:05:56 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 19:05:56 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> On Wednesday, February 11, 2015 at 2:36:23 AM UTC+5:30, Skip Montanaro wrote: > I know this is way off-topic for this group, but I figured if anyone > in the online virtual communities I participate in would know the > answer, the Pythonistas would... Google has so far not been my friend > in this realm. > > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. For > instance, If I press and hold the "d" key, I see these choices (ignore > the capitalization of the first letter - my mistake sending a text > message to myself from my phone, and I can't seem to convert it to > lower case): ?|?&d?? > > While I'm a touch typist, I almost never use auto-repeat, which is the > "binding" of held keys in most environments (curse you, IBM and your > Selectric!). These days I find my self needing accented characters > much more frequently than key repeat (C-u 2 5 - suffices in Emacs to > bat out 25 hyphens). Being an American with an American keyboard, I > haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard, and no key > caps display what the various options are. And I'm getting kind of > tired of going to Google and searching for "degree symbol". :-/ > > Is there an X11 or Mac extension/program/app/magic thing which I can > install in either environment to get this kind of functionality? I'm > thinking that if you hold down a key for the auto-repeat interval, > instead of the key repeat thing making all sorts of duplicates, a > little window would pop up over/near the insertion point, which I can > navigate with the arrow keys, then hit RET to accept or ESC (or > similar) to cancel. It need not be perfect. It might (for example) > only work in certain environments (Chrome, Emacs, vim, Firefox). > Anyplace to start. It need even be written in Python (though that > would be cool.) I think that once something like this caught hold, it > would fairly quickly take over from the dark lords of auto-repeat. > > Thx, > > Skip Nice question ? I too await an answer. Was amused when gmail offered facilities to type devanagari. After using for a while found it way too clever for my taste and found emacs' itrans input method better ? emacs is slightly dumb, gmail is way too clever. Here is (Yuri Khan's answers) about changing Xorg keyboard maps: https://groups.google.com/forum/#!searchin/gnu.emacs.help/Yuri$20X$20Rusi/gnu.emacs.help/yesOU0m0vIE/CEvhlRZZY6kJ Here is a rather neat general collection of compose settings: https://github.com/rrthomas/pointless-xcompose? > And I'm getting kind of tired of going to Google and searching for "degree symbol". :-/ Yeah? More to add to your question than to answer: my recent blog post: blog.languager.org/2015/01/unicode-and-universe.html Towards the end there are multi-levels of input methods. It would be good to work out in greater detail the intermediate levels between "google for degree symbol" and "type degree symbol on keyboard" ? He does not explain how to setup compose though the whole point of pointless(!) is to use it. $ setxkbmap -option compose:menu # set compose to menu (Windows-menu) $ setxkbmap -option compose:rwin # to rwin $ setxkbmap -option compose:ralt # to ralt (the usual AltGr) $ setxkbmap -query # examine current settings $ setxkbmap -option # No option -- turns off all options From nad at acm.org Tue Feb 10 22:37:27 2015 From: nad at acm.org (Ned Deily) Date: Tue, 10 Feb 2015 19:37:27 -0800 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: Message-ID: In article , Skip Montanaro wrote: > I know this is way off-topic for this group, but I figured if anyone > in the online virtual communities I participate in would know the > answer, the Pythonistas would... Google has so far not been my friend > in this realm. > > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. For > instance, If I press and hold the "d" key, I see these choices (ignore > the capitalization of the first letter - my mistake sending a text > message to myself from my phone, and I can't seem to convert it to > lower case): ?|?&d??? > > While I'm a touch typist, I almost never use auto-repeat, which is the > "binding" of held keys in most environments (curse you, IBM and your > Selectric!). These days I find my self needing accented characters > much more frequently than key repeat (C-u 2 5 - suffices in Emacs to > bat out 25 hyphens). Being an American with an American keyboard, I > haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard, and no key > caps display what the various options are. And I'm getting kind of > tired of going to Google and searching for "degree symbol". :-/ > > Is there an X11 or Mac extension/program/app/magic thing which I can > install in either environment to get this kind of functionality? I'm > thinking that if you hold down a key for the auto-repeat interval, > instead of the key repeat thing making all sorts of duplicates, a > little window would pop up over/near the insertion point, which I can > navigate with the arrow keys, then hit RET to accept or ESC (or > similar) to cancel. It need not be perfect. It might (for example) > only work in certain environments (Chrome, Emacs, vim, Firefox). > Anyplace to start. It need even be written in Python (though that > would be cool.) I think that once something like this caught hold, it > would fairly quickly take over from the dark lords of auto-repeat. On OS X, the system provides both a "Character Viewer" (which allows the selection of any Unicode character and a "Keyboard Viewer" (which allows you to see the keyboard key mappings as keys and modifiers are pressed). Both can be enabled from System Preferences (System Preferences -> Keyboard -> Keyboard on current 10.10 systems). As of OS X 10.7 (I believe), there are also now popup menus that appear when various keys are held down that show accented variants of the character (this is similar to what iOS on the iPhone and iPad provide). Unfortunately, this doesn't work with some applicatons that do not use the full built-in text system; the Tk implementations on OS X are such applications so the popups don't work with Tkinter apps like IDLE. Nor does it work with X11 apps. -- Ned Deily, nad at acm.org From steve+comp.lang.python at pearwood.info Tue Feb 10 23:10:36 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 11 Feb 2015 15:10:36 +1100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> Message-ID: <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> Rustom Mody wrote: > $ setxkbmap -query # examine current settings Alas, that does not appear to work in Debian squeeze: steve at runes:~$ setxkbmap -query Error! Option "-query" not recognized Or Centos. What are you using? -- Steve From rosuav at gmail.com Tue Feb 10 23:19:59 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 15:19:59 +1100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wed, Feb 11, 2015 at 3:10 PM, Steven D'Aprano wrote: > Rustom Mody wrote: > >> $ setxkbmap -query # examine current settings > > Alas, that does not appear to work in Debian squeeze: > > steve at runes:~$ setxkbmap -query > Error! Option "-query" not recognized > > Or Centos. > > What are you using? Works for me on Debian Wheezy. Either it's a version difference (I don't have any Squeeze machines any more, we're all on Wheezy or Jessie), or there's some additional package that I have installed here. Mind you, I have no idea what I'm looking at. rosuav at sikorsky:~$ setxkbmap -query rules: evdev model: pc105 layout: us I guess that means I have a 105-key PC keyboard in en_US layout, but what 'evdev' means I would have to go manpage digging to find out. ChrisA From rustompmody at gmail.com Tue Feb 10 23:23:37 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 20:23:37 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> Message-ID: <1b7f372a-537a-452d-99cb-b60c1748cccb@googlegroups.com> On Wednesday, February 11, 2015 at 9:40:50 AM UTC+5:30, Steven D'Aprano wrote: > Rustom Mody wrote: > > > $ setxkbmap -query # examine current settings > > Alas, that does not appear to work in Debian squeeze: > > steve at runes:~$ setxkbmap -query > Error! Option "-query" not recognized > > Or Centos. > > What are you using? $ dpkg -S setxkbmap x11-xkb-utils: /usr/bin/setxkbmap [and some doc stuff] So... $ aptitude show x11-xkb-utils|grep Version Version: 7.7+1 [which is probably what you need to know] To answer more literally: Just now this is a debian testing running xfce I believe this works also on ubuntu 14.10 From rosuav at gmail.com Tue Feb 10 23:38:10 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 11 Feb 2015 15:38:10 +1100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <1b7f372a-537a-452d-99cb-b60c1748cccb@googlegroups.com> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <1b7f372a-537a-452d-99cb-b60c1748cccb@googlegroups.com> Message-ID: On Wed, Feb 11, 2015 at 3:23 PM, Rustom Mody wrote: > $ aptitude show x11-xkb-utils|grep Version > Version: 7.7+1 > > [which is probably what you need to know] Tip: Getting version info can be done less spammily with apt-cache policy. rosuav at sikorsky:~$ apt-cache policy x11-xkb-utils x11-xkb-utils: Installed: 7.7~1 Candidate: 7.7~1 Version table: *** 7.7~1 0 500 http://ftp.au.debian.org/debian/ wheezy/main amd64 Packages 100 /var/lib/dpkg/status ChrisA From rustompmody at gmail.com Tue Feb 10 23:42:32 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 20:42:32 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> Message-ID: <7205e8e1-8daf-4b6d-a54f-2a9584846ed5@googlegroups.com> On Wednesday, February 11, 2015 at 9:50:15 AM UTC+5:30, Chris Angelico wrote: > On Wed, Feb 11, 2015 at 3:10 PM, Steven D'Aprano wrote: > > Rustom Mody wrote: > > > >> $ setxkbmap -query # examine current settings > > > > Alas, that does not appear to work in Debian squeeze: > > > > steve at runes:~$ setxkbmap -query > > Error! Option "-query" not recognized > > > > Or Centos. > > > > What are you using? > > Works for me on Debian Wheezy. Either it's a version difference (I > don't have any Squeeze machines any more, we're all on Wheezy or > Jessie), or there's some additional package that I have installed > here. This xkb stuff seems to be (surprisingly) in development -- if only the docs kept pace! eg My impression is that ubuntus prior to 11.x did not have apl keyboard Somewhere around 11/12 or after this works setxkbmap -layout "us,apl" -option "grp:switch" after that abcdefghijklmnopqrstuvwxyz produces ?????_????'?|??*???~?????? when chorded with R-Alt If that looks gibberish see that keyboard: http://xahlee.info/kbd/creating_apl_keyboard_layout.html Replace the 'apl' in the setxkb by 'gr' and you will see more familiar territory ????????????????;????????? ????????????????:???????? Of course for it to work the alternate should be working. eg right now on my box: $ setxkbmap -query rules: evdev model: pc104 layout: us,gr options: grp:switch,compose:menu which shows gr as alternate and the grp:switch (whatever that means) From lk.palao at gmail.com Wed Feb 11 00:11:30 2015 From: lk.palao at gmail.com (Leo Kris Palao) Date: Wed, 11 Feb 2015 13:11:30 +0800 Subject: GDAL installation Message-ID: Hi Python Users, I currently installed the Python 2.7.9 and installed the GDAL package. First, I tried to install GDAL using PIP but it throws an error - I cannot remember the exact error message. So, I install it using easy_install command. But when I import the package I am getting this message, which I really don't understand. Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\lpalao>python > Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit (AMD64)] > on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import gdal > Traceback (most recent call last): > File "", line 1, in > File "C:\Python27\Python2.7.9\lib\site-packages\gdal.py", line 2, in > > from osgeo.gdal import deprecation_warn > File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", line > 21, in > _gdal = swig_import_helper() > File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", line > 17, in swig_import_helper > _mod = imp.load_module('_gdal', fp, pathname, description) > ImportError: DLL load failed: The specified module could not be found. > >>> Thanks in advance, -Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From kushal at locationd.net Wed Feb 11 01:15:44 2015 From: kushal at locationd.net (Kushal Kumaran) Date: Wed, 11 Feb 2015 11:45:44 +0530 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: (Skip Montanaro's message of "Tue, 10 Feb 2015 15:05:47 -0600") References: Message-ID: <87twyt2ctr.fsf@carbon.locationd.net.> Skip Montanaro writes: > I know this is way off-topic for this group, but I figured if anyone > in the online virtual communities I participate in would know the > answer, the Pythonistas would... Google has so far not been my friend > in this realm. > > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. For > instance, If I press and hold the "d" key, I see these choices (ignore > the capitalization of the first letter - my mistake sending a text > message to myself from my phone, and I can't seem to convert it to > lower case): ?|?&d?? > > While I'm a touch typist, I almost never use auto-repeat, which is the > "binding" of held keys in most environments (curse you, IBM and your > Selectric!). These days I find my self needing accented characters > much more frequently than key repeat (C-u 2 5 - suffices in Emacs to > bat out 25 hyphens). Being an American with an American keyboard, I > haven't the slightest idea how to type any accented characters or > common symbols using the many modifier keys on my keyboard, and no key > caps display what the various options are. And I'm getting kind of > tired of going to Google and searching for "degree symbol". :-/ > > Is there an X11 or Mac extension/program/app/magic thing which I can > install in either environment to get this kind of functionality? I'm > thinking that if you hold down a key for the auto-repeat interval, > instead of the key repeat thing making all sorts of duplicates, a > little window would pop up over/near the insertion point, which I can > navigate with the arrow keys, then hit RET to accept or ESC (or > similar) to cancel. It need not be perfect. It might (for example) > only work in certain environments (Chrome, Emacs, vim, Firefox). > Anyplace to start. It need even be written in Python (though that > would be cool.) I think that once something like this caught hold, it > would fairly quickly take over from the dark lords of auto-repeat. > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). >From it's documentation: Interactively, prompt for CHARACTER. You can specify CHARACTER in one of these ways: - As its Unicode character name, e.g. "LATIN SMALL LETTER A". Completion is available; if you type a substring of the name preceded by an asterisk `*', Emacs shows all names which include that substring, not necessarily at the beginning of the name. - As a hexadecimal code point, e.g. 263A. Note that code points in Emacs are equivalent to Unicode up to 10FFFF (which is the limit of the Unicode code space). - As a code point with a radix specified with #, e.g. #o21430 (octal), #x2318 (hex), or #10r8984 (decimal). For your "degree symbol" example, I did C-x 8 RET, then typed in *degree, and emacs found some matches, resulting in this: ? -- regards, kushal From andrew at geekuni.com Wed Feb 11 01:33:32 2015 From: andrew at geekuni.com (Andrew Solomon) Date: Wed, 11 Feb 2015 06:33:32 +0000 Subject: State of the Geek Survey - Win a $50 Amazon Gift Card! Message-ID: Hello Python Developers, Complete our survey on the future of languages for the chance to win a $50 Amazon Gift Card https://geekuni.com/survey.html Thanks! Andrew p.s. The data (minus emails) will be publicly available. From rustompmody at gmail.com Wed Feb 11 02:28:02 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 10 Feb 2015 23:28:02 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: On Wednesday, February 11, 2015 at 11:55:19 AM UTC+5:30, Kushal Kumaran wrote: > Skip Montanaro writes: > > > I know this is way off-topic for this group, but I figured if anyone > > in the online virtual communities I participate in would know the > > answer, the Pythonistas would... Google has so far not been my friend > > in this realm. > > > > One of the things I really like about my Skype keyboard (and likely > > other "soft" keyboards on Android) is that when you hold down a "key" > > for a brief moment, a little mini keyboard pops up, from which you can > > easily choose various accented variants and other symbols. For > > instance, If I press and hold the "d" key, I see these choices (ignore > > the capitalization of the first letter - my mistake sending a text > > message to myself from my phone, and I can't seem to convert it to > > lower case): ?|?&d?? > > > > While I'm a touch typist, I almost never use auto-repeat, which is the > > "binding" of held keys in most environments (curse you, IBM and your > > Selectric!). These days I find my self needing accented characters > > much more frequently than key repeat (C-u 2 5 - suffices in Emacs to > > bat out 25 hyphens). Being an American with an American keyboard, I > > haven't the slightest idea how to type any accented characters or > > common symbols using the many modifier keys on my keyboard, and no key > > caps display what the various options are. And I'm getting kind of > > tired of going to Google and searching for "degree symbol". :-/ > > > > Is there an X11 or Mac extension/program/app/magic thing which I can > > install in either environment to get this kind of functionality? I'm > > thinking that if you hold down a key for the auto-repeat interval, > > instead of the key repeat thing making all sorts of duplicates, a > > little window would pop up over/near the insertion point, which I can > > navigate with the arrow keys, then hit RET to accept or ESC (or > > similar) to cancel. It need not be perfect. It might (for example) > > only work in certain environments (Chrome, Emacs, vim, Firefox). > > Anyplace to start. It need even be written in Python (though that > > would be cool.) I think that once something like this caught hold, it > > would fairly quickly take over from the dark lords of auto-repeat. > > > > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). For slightly more frequent use (emacs-users) 1. Find a symbol such as ? maybe using google and paste it into emacs Does it exist in tex input method? To find out 2. Turn on tex input method (in some buffer) ie C-x RET C-\ and give tex (or whatever 3. C-u C-x = will tell info about the char. In particular it says: To input type \degree in tex input method I guess for diacritical marks one could use one of the latin-n input methods The above applied to ? and latin-2-postfix says that ? can be input with a' M-x describe-input-method will tell you others like ------------+---------+---------- acute | ' | a' -> ? ogonek | , | a, -> ? diaeresis | " | a" -> ? circumflex | ^ | a^ -> ? breve | ~ | a~ -> ? cedilla | , | c, -> ? caron | ~ | c~ -> ? dbl. acute | : | o: -> ? ring | . | u. -> ? dot | . | z. -> ? stroke | / | d/ -> ? others | / | s/ -> ? From marko at pacujo.net Wed Feb 11 02:31:09 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Feb 2015 09:31:09 +0200 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: Message-ID: <87iof8c3b6.fsf@elektro.pacujo.net> Kushal Kumaran : > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). Emacs to the rescue, as usual. I use emacs for all of my typing needs. If I stray on a Web form, I end up cussing at the browser whenever I instinctively hit a C-n, C-a or C-s (or, yesterday, C-w). As for typing "special" characters, I have written a .Xmodmap file that rebinds my keyboard for various languages I use regularly. Then, I have this text file that contains specimens for many other useful characters. Marko From wxjmfauth at gmail.com Wed Feb 11 03:12:19 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 11 Feb 2015 00:12:19 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <87iof8c3b6.fsf@elektro.pacujo.net> References: <87iof8c3b6.fsf@elektro.pacujo.net> Message-ID: Le mercredi 11 f?vrier 2015 08:31:21 UTC+1, Marko Rauhamaa a ?crit?: > Kushal Kumaran : > > > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). > > Emacs to the rescue, as usual. > > I use emacs for all of my typing needs. If I stray on a Web form, I end > up cussing at the browser whenever I instinctively hit a C-n, C-a or C-s > (or, yesterday, C-w). > > As for typing "special" characters, I have written a .Xmodmap file that > rebinds my keyboard for various languages I use regularly. Then, I have > this text file that contains specimens for many other useful characters. > > > Marko In my interactive Python interpreter, I can enter all these chars (any unicode char) directly from the main keyoard (a lot of work). Python 3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)] on win32 >>> eta runs etazero.py... ...etazero has been executed >>> print('abc??????????') abc?????????? >>> As you can see, it's Python 32. Rusy: The last char is a Devanagari char. jmf From asimjalis at gmail.com Wed Feb 11 03:48:02 2015 From: asimjalis at gmail.com (Asim Jalis) Date: Wed, 11 Feb 2015 00:48:02 -0800 Subject: GDAL installation In-Reply-To: References: Message-ID: Hi Leo, This might be a PATH issue. See this discussion for details. https://pythongisandstuff.wordpress.com/2011/07/07/installing-gdal-and-ogr-for-python-on-windows/ Asim On Tue, Feb 10, 2015 at 9:11 PM, Leo Kris Palao wrote: > Hi Python Users, > > I currently installed the Python 2.7.9 and installed the GDAL package. > First, I tried to install GDAL using PIP but it throws an error - I cannot > remember the exact error message. So, I install it using easy_install > command. But when I import the package I am getting this message, which I > really don't understand. > > Microsoft Windows [Version 6.1.7601] > Copyright (c) 2009 Microsoft Corporation. All rights reserved. > > C:\Users\lpalao>python >> Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit (AMD64)] >> on win32 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import gdal >> Traceback (most recent call last): >> File "", line 1, in >> File "C:\Python27\Python2.7.9\lib\site-packages\gdal.py", line 2, in >> >> from osgeo.gdal import deprecation_warn >> File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", >> line 21, in >> _gdal = swig_import_helper() >> File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", >> line 17, in swig_import_helper >> _mod = imp.load_module('_gdal', fp, pathname, description) >> ImportError: DLL load failed: The specified module could not be found. >> >>> > > > Thanks in advance, > -Leo > > -- > https://mail.python.org/mailman/listinfo/python-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wxjmfauth at gmail.com Wed Feb 11 03:49:43 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 11 Feb 2015 00:49:43 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: <87iof8c3b6.fsf@elektro.pacujo.net> Message-ID: <3517be7e-b550-4399-ba64-7630304b2b85@googlegroups.com> Le mercredi 11 f?vrier 2015 09:12:30 UTC+1, wxjm... at gmail.com a ?crit?: > Le mercredi 11 f?vrier 2015 08:31:21 UTC+1, Marko Rauhamaa a ?crit?: > > Kushal Kumaran : > > > > > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). > > > > Emacs to the rescue, as usual. > > > > I use emacs for all of my typing needs. If I stray on a Web form, I end > > up cussing at the browser whenever I instinctively hit a C-n, C-a or C-s > > (or, yesterday, C-w). > > > > As for typing "special" characters, I have written a .Xmodmap file that > > rebinds my keyboard for various languages I use regularly. Then, I have > > this text file that contains specimens for many other useful characters. > > > > > > Marko > > In my interactive Python interpreter, I can > enter all these chars (any unicode char) directly from > the main keyoard (a lot of work). > > Python 3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)] on win32 > >>> eta runs etazero.py... > ...etazero has been executed > >>> print('abc??????????') > abc?????????? > >>> > > As you can see, it's Python 32. > Rusy: The last char is a Devanagari char. > > jmf Just a check: It was sent via Google group, it is rendered correctly. Addendum: bidi example, Arabic and Hebrew letters. >>> print('ab????xyz') ab????xyz Hebrew works fine, not so sure about arabic text. From wxjmfauth at gmail.com Wed Feb 11 04:01:36 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 11 Feb 2015 01:01:36 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <599f6b57-5791-4a08-8fb2-4272a4b31d90@googlegroups.com> Le mardi 10 f?vrier 2015 22:06:23 UTC+1, Skip Montanaro a ?crit?: > > One of the things I really like about my Skype keyboard (and likely > other "soft" keyboards on Android) is that when you hold down a "key" > for a brief moment, a little mini keyboard pops up, from which you can > easily choose various accented variants and other symbols. For > instance, If I press and hold the "d" key, I see these choices (ignore > the capitalization of the first letter - my mistake sending a text > message to myself from my phone, and I can't seem to convert it to > lower case): ?|?&d?? > I purchased a tablet very recently (1 year) and my first thought was "Ah, there are people who are understanding Unicode". I'm not laying, I'm able to implement something like this in a GUI application. Technically it's obvious (for me). jmf From wxjmfauth at gmail.com Wed Feb 11 04:17:52 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 11 Feb 2015 01:17:52 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <3517be7e-b550-4399-ba64-7630304b2b85@googlegroups.com> References: <87iof8c3b6.fsf@elektro.pacujo.net> <3517be7e-b550-4399-ba64-7630304b2b85@googlegroups.com> Message-ID: Le mercredi 11 f?vrier 2015 09:49:57 UTC+1, wxjm... at gmail.com a ?crit?: > Le mercredi 11 f?vrier 2015 09:12:30 UTC+1, wxjm... at gmail.com a ?crit?: > > Le mercredi 11 f?vrier 2015 08:31:21 UTC+1, Marko Rauhamaa a ?crit?: > > > Kushal Kumaran : > > > > > > > For very, very occasional use in emacs, there's C-x 8 RET (insert-char). > > > > > > Emacs to the rescue, as usual. > > > > > > I use emacs for all of my typing needs. If I stray on a Web form, I end > > > up cussing at the browser whenever I instinctively hit a C-n, C-a or C-s > > > (or, yesterday, C-w). > > > > > > As for typing "special" characters, I have written a .Xmodmap file that > > > rebinds my keyboard for various languages I use regularly. Then, I have > > > this text file that contains specimens for many other useful characters. > > > > > > > > > Marko > > > > In my interactive Python interpreter, I can > > enter all these chars (any unicode char) directly from > > the main keyoard (a lot of work). > > > > Python 3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)] on win32 > > >>> eta runs etazero.py... > > ...etazero has been executed > > >>> print('abc??????????') > > abc?????????? > > >>> > > > > As you can see, it's Python 32. > > Rusy: The last char is a Devanagari char. > > > > jmf > > > Just a check: It was sent via Google group, it is rendered > correctly. > > Addendum: bidi example, Arabic and Hebrew letters. > > >>> print('ab????xyz') > ab????xyz > > Hebrew works fine, not so sure about arabic text. Ok, it is also rendered correctly. After all, this google group is not so bad. jmf From alister.nospam.ware at ntlworld.com Wed Feb 11 04:33:52 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Wed, 11 Feb 2015 09:33:52 +0000 (UTC) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <20150210152900.54129700@bigbox.christie.dr> Message-ID: On Tue, 10 Feb 2015 23:15:11 +0100, Laura Creighton wrote: > In a message of Tue, 10 Feb 2015 15:29:00 -0600, Tim Chase writes: >>While it's not exactly a hold-down-get-a-menu, I opt for changing my >>(otherwise-useless) caps-lock key to an X compose key: >> >> $ setxkbmap -option compose:caps >> >>I can then hit caps-lock followed by what are generally intuitive >>sequences. For your first one, that would be "capital-D minus". I'm >>not sure what the other characters are supposed to be, so I'm not sure >>how to find them. But ? is "compose, e, apostrophe", ? is "compose, n, >>tilde", the degree sign is "compose, o, o", the ? is "compose, E, >>equals", etc. There are loads of these documented in (on my machine, >>where my locale is en_US.UTF-8) >>/usr/share/X11/locale/en_US.UTF-8/Compose >> >>Some of them are a little less intuitive, though the majority of the >>time I can just guess them (I'd never typed "?" before, but guessed and >>was right). Otherwise I search that above file. >> >>This also has the advantage that it should work in every X application, >>including Unicode-aware terminal applications (in Unicode-aware >>terminals). Adding some sort of press-and-hold UI would limit it to >>those applications that chose to support it (or even *could* support >>it). >> >>> While I'm a touch typist, I almost never use auto-repeat, which is the >>> "binding" of held keys in most environments >> >>I agree, as vi/vim makes it easy to insert multiples of the same >>character (or characters) akin to what you describe in Emacs. >> >>-tkc > > Wow. US keyboards do not come with a 'compose' key, then? It just > never occurred to me that Skip might be missing one. > > Oh, goodness gracious then, go with this solution. Much better than > mine --though the one I pointed at is great should you suddenly need to > type something in cyrillic while at a non-cyrillic keyboard. > > Laura English keyboards (US & UK) don't have a compose key because wee do not normaly use accented characters (except when dealing with people that do & then being lazy we often cheat & don't bother with them. Apologies to all who do use them this is very poor behaviour) -- No character, however upright, is a match for constantly reiterated attacks, however false. -- Alexander Hamilton From brunocauet at gmail.com Wed Feb 11 05:03:55 2015 From: brunocauet at gmail.com (Bruno Cauet) Date: Wed, 11 Feb 2015 11:03:55 +0100 Subject: urllib.parser.quote() and RFC 2396: unreserved characters get encoded Message-ID: Hi, I believe that pathlib and urllib.parse.quote() do not correctly build URIs. According to RFC 2396 (http://tools.ietf.org/html/rfc2396.html), unreserved characters such as "!" should not be escaped (section 2.3): Unreserved characters can be escaped without changing the semantics of the URI, but this should not be done unless the URI is being used in a context that does not allow the unescaped character to appear. Unicode characters outside the ASCII range also get encoded when they have no reason to, e.g. >>> pathlib.PurePath("/home/??/").as_uri() 'file:///home/%EC%8B%B8%EC%9D%B4' while the result should simply be 'file:///home/??' The documentation (https://docs.python.org/3.5/library/urllib.parse.html) mentions RFC 2396, so I think it should be respected. Am I missing something? Is it a bug? Source-wise, the problem stems from usage of urllib.parse.quote_from_bytes() which quotes everything. I'm not sure whether the responsibility lies, but I believe that urllib.parse.quote() is affected by the same problem, even though it mentions RFC 2396 in its docstring. The problem I have because of this is that URIs generated by python does not match the URIs generated by other tools and make me unable to identify elements as identical because of their URI. Thanks, Bruno From alec.taylor6 at gmail.com Wed Feb 11 06:15:04 2015 From: alec.taylor6 at gmail.com (Alec Taylor) Date: Wed, 11 Feb 2015 03:15:04 -0800 (PST) Subject: JDBC support in a CPython ORM? - Maybe via the JayDeBeApi package? Message-ID: <2bee15bb-1a6d-489b-80f9-5606edf120c3@googlegroups.com> Attempting to connect to Hadoop-based SQL layers, which are [practically] all written in JVM languages. Sure, I can use something like JayDeBeApi, but I would prefer proper object mapping. How would I go about interfacing with e.g.: SQL Alchemy or peewee-orm? Thanks for all suggestions From rustompmody at gmail.com Wed Feb 11 07:22:38 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 04:22:38 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> Message-ID: <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> On Wednesday, February 11, 2015 at 9:50:15 AM UTC+5:30, Chris Angelico wrote: > On Wed, Feb 11, 2015 at 3:10 PM, Steven D'Aprano wrote: > > Rustom Mody wrote: > > > >> $ setxkbmap -query # examine current settings > > > > Alas, that does not appear to work in Debian squeeze: > > > > steve at runes:~$ setxkbmap -query > > Error! Option "-query" not recognized > > > > Or Centos. > > > > What are you using? > > Works for me on Debian Wheezy. Either it's a version difference (I > don't have any Squeeze machines any more, we're all on Wheezy or > Jessie), or there's some additional package that I have installed > here. > > Mind you, I have no idea what I'm looking at. > > rosuav at sikorsky:~$ setxkbmap -query > rules: evdev > model: pc105 > layout: us > > I guess that means I have a 105-key PC keyboard in en_US layout, but > what 'evdev' means I would have to go manpage digging to find out. > > ChrisA Small request (especially if you have an older X): Can someone try out: $ setxkbmap -layout "us,gr" -option "grp:ralt_rshift_toggle" Earlier that used to work as a *toggle* ie a RAlt-RShift chord would go to gr(eek) and another would come back to 'us'. Now it seems (on my box) that it goes to gr(eek) BUT NOT BACK. NOTE: that this means one has effectively a broken keyboard... Until you do something like: $ setxkbmap -layout "us" -option Getting the first from the second should of course work if shell history -- ie up-arrow -- and backspace work. From marko at pacujo.net Wed Feb 11 07:35:49 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Feb 2015 14:35:49 +0200 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> Message-ID: <874mqsbp7e.fsf@elektro.pacujo.net> Rustom Mody : > Can someone try out: > > $ setxkbmap -layout "us,gr" -option "grp:ralt_rshift_toggle" I tried it. > Earlier that used to work as a *toggle* ie a RAlt-RShift chord would > go to gr(eek) and another would come back to 'us'. I noticed, thanks very much. Luckily I managed to undo the damage without logging out. The reason is the Greek keyboard doesn't have an RAlt. Solution: $ setxkbmap -layout "us,gr" -option "grp:lalt_rshift_toggle" The problem with xkbmap is that I don't know how to specify a new keyboard map as a regular user. I know how to do that with .Xmodmap. Marko From rustompmody at gmail.com Wed Feb 11 07:40:54 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 04:40:54 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> Message-ID: <107cf10a-9bd0-46a2-8c43-8e0ca77146df@googlegroups.com> On Wednesday, February 11, 2015 at 5:52:49 PM UTC+5:30, Rustom Mody wrote: > On Wednesday, February 11, 2015 at 9:50:15 AM UTC+5:30, Chris Angelico wrote: > > On Wed, Feb 11, 2015 at 3:10 PM, Steven D'Aprano wrote: > > > Rustom Mody wrote: > > > > > >> $ setxkbmap -query # examine current settings > > > > > > Alas, that does not appear to work in Debian squeeze: > > > > > > steve at runes:~$ setxkbmap -query > > > Error! Option "-query" not recognized > > > > > > Or Centos. > > > > > > What are you using? > > > > Works for me on Debian Wheezy. Either it's a version difference (I > > don't have any Squeeze machines any more, we're all on Wheezy or > > Jessie), or there's some additional package that I have installed > > here. > > > > Mind you, I have no idea what I'm looking at. > > > > rosuav at sikorsky:~$ setxkbmap -query > > rules: evdev > > model: pc105 > > layout: us > > > > I guess that means I have a 105-key PC keyboard in en_US layout, but > > what 'evdev' means I would have to go manpage digging to find out. > > > > ChrisA > > Small request (especially if you have an older X): > > Can someone try out: > > $ setxkbmap -layout "us,gr" -option "grp:ralt_rshift_toggle" > > Earlier that used to work as a *toggle* ie a RAlt-RShift chord would go to gr(eek) > and another would come back to 'us'. > > Now it seems (on my box) that it goes to gr(eek) BUT NOT BACK. > > NOTE: that this means one has effectively a broken keyboard... > > Until you do something like: > > $ setxkbmap -layout "us" -option > > Getting the first from the second should of course work if shell history -- ie > up-arrow -- and backspace work. Just checked that $ setxkbmap -layout "us,gr" -option "grp:caps_toggle,grp_led:caps" will convert CapsLock into a 'GreekLock' ie after a CapsLock abcdefg produces ??????? ABCDEFG produces ??????? (ie use Shift as usual but after capslock) and 1. the CapsLock led helpfully indicates the state 2. It toggles back!! If someone can make the RAlt-RShift chord work like capslock please let me know From rustompmody at gmail.com Wed Feb 11 07:57:07 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 04:57:07 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <874mqsbp7e.fsf@elektro.pacujo.net> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> Message-ID: <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> On Wednesday, February 11, 2015 at 6:06:03 PM UTC+5:30, Marko Rauhamaa wrote: > The problem with xkbmap is that I don't know how to specify a new > keyboard map as a regular user. I know how to do that with .Xmodmap. Xmodmap is neat... but obsolete Ive noticed hanging and misbehavior and there are all sorts of bugs eg https://bugs.launchpad.net/ubuntu/+source/linux/+bug/998310 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=550533 all of which amount to "Dont use Xmodmap; use Xkb" Yeah as I said Xmodmap is much simpler than xkb but the reality is that former is obsolete and increasingly unsupported. > Rustom Mody wrote: > > > Can someone try out: > > > > $ setxkbmap -layout "us,gr" -option "grp:ralt_rshift_toggle" > > I tried it. > > > Earlier that used to work as a *toggle* ie a RAlt-RShift chord would > > go to gr(eek) and another would come back to 'us'. > > I noticed, thanks very much. :-) > Luckily I managed to undo the damage without logging out. > > The reason is the Greek keyboard doesn't have an RAlt. Solution: > > $ setxkbmap -layout "us,gr" -option "grp:lalt_rshift_toggle" Thanks for trying (and explaining). So a bit wiser now I see that $ setxkbmap -layout "us,gr" -option "grp:lalt_lshift_toggle,grp_led:caps" toggles with lalt-lshift and shows indicator on caps-led $ setxkbmap -layout "us,gr" -option "grp:lalt_lshift_toggle,grp_led:scroll" toggles with lalt-lshift and shows indicator on scroll-led From albert at spenarnc.xs4all.nl Wed Feb 11 08:04:22 2015 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 11 Feb 2015 13:04:22 GMT Subject: Alternative to multi-line lambdas: Assign-anywhere def statements References: <54C3EAD1.2010501@stoneleaf.us> Message-ID: <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> In article , Ethan Furman wrote: >-=-=-=-=-=- > >On 01/24/2015 11:55 AM, Chris Angelico wrote: >> On Sun, Jan 25, 2015 at 5:56 AM, Ethan Furman wrote: >>> If the non-generic is what you're concerned about: >>> >>> # not tested >>> dispatch_table_a = {} >>> dispatch_table_b = {} >>> dispatch_table_c = {} >>> >>> class dispatch: >>> def __init__(self, dispatch_table): >>> self.dispatch = dispatch_table >>> def __call__(self, func): >>> self.dispatch[func.__name__] = func >>> return func >>> >>> @dispatch(dispatch_table_a) >>> def foo(...): >>> pass >> >> That's still only able to assign to a key of a dictionary, using the >> function name. > >This is a Good Thing. The def statement populates a few items, __name__ being one of them. One >of the reasons lambda >is not encouraged is because its name is always '', which just ain't helpful when the >smelly becomes air borne! ;) That's the reason why my ideal Python doesn't attach a name to a lambda denotation: x -> x**2 is a function object, not something that has a name. It is not until we assign the object to a name (which becomes thereby a function) that the __name__ thingy comes into play, like so. f = x->x**2 or f = x-> return x**2 for those who don't like Algol68 I've heard arguments that with -> the __name__ is not filled in correctly. I can't see why the parser would understand more easily def f(x): return x**2 than f = x-> return x**2 [I'm striving for simplification, doing away with both the lambda and the def keywords. This is not a proposal for a language change, I'm trying to explore possibilities here. ] > >-- >~Ethan~ Groetjes Albert -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From python.list at tim.thechases.com Wed Feb 11 08:04:55 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 11 Feb 2015 07:04:55 -0600 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: <20150211070455.5be5b83c@bigbox.christie.dr> On 2015-02-10 19:37, Ned Deily wrote: > On OS X, the system provides both a "Character Viewer" (which > allows the selection of any Unicode character Windows also provides charmap.exe which provides similar functionality, though last I checked it, it still had the feel of a Win3.1 app (usability was atrocious) -tkc From rosuav at gmail.com Wed Feb 11 08:09:35 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Feb 2015 00:09:35 +1100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <20150211070455.5be5b83c@bigbox.christie.dr> References: <20150211070455.5be5b83c@bigbox.christie.dr> Message-ID: On Thu, Feb 12, 2015 at 12:04 AM, Tim Chase wrote: > On 2015-02-10 19:37, Ned Deily wrote: >> On OS X, the system provides both a "Character Viewer" (which >> allows the selection of any Unicode character > > Windows also provides charmap.exe which provides similar > functionality, though last I checked it, it still had the feel of a > Win3.1 app (usability was atrocious) > Back in the Win3.1 days, you could make a character map program that was reasonably compact - it had to offer you just 256 characters (and maybe allow you to choose a code page). Now, it's a lot broader than that... but on the flip side, I can make a single program that manipulates a single text document that contains text in all of the world's languages, so it's well worth it. And while I have never actually used _all_ the world's languages at once, I have at least gone as far as a couple dozen. ChrisA From rustompmody at gmail.com Wed Feb 11 08:11:08 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 05:11:08 -0800 (PST) Subject: idle feature request Message-ID: Context: I am using idle for taking python classes. Finish the class and run out usually in a hurry and forget to save the idle interaction window. Would like to save it so that I can mail it to the students. In emacs I could set a hook to make arbitrary 'buffers' like the python-idle shell become 'saving-buffers' like typical file buffers. ie in emacs like most other editors, a modified file is ask-to-save The same can be setup for other (non-file) buffers like a python interaction shell Can I do the same for idle. [Note it would be great to have this feature in idle but I am only looking for a hand in where in the idle sources I an fudge a patch for my own use ] From fabien.maussion at gmail.com Wed Feb 11 08:20:21 2015 From: fabien.maussion at gmail.com (Fabien) Date: Wed, 11 Feb 2015 14:20:21 +0100 Subject: idle feature request References: Message-ID: On 11.02.2015 14:11, Rustom Mody wrote: > Context: > I am using idle for taking python classes. I know this is not your question, but: have you considered using iPython notebooks for teaching? They save automatically, look nice, and can be sent as code, html, rst, ... Fabien From vlyamtsev at gmail.com Wed Feb 11 08:27:26 2015 From: vlyamtsev at gmail.com (Victor L) Date: Wed, 11 Feb 2015 08:27:26 -0500 Subject: function inclusion problem In-Reply-To: <201502110016.t1B0GVah024279@fido.openend.se> References: <201502110006.t1B060VZ021974@fido.openend.se> <201502110016.t1B0GVah024279@fido.openend.se> Message-ID: Laura, thanks for the answer - it works. Is there some equivalent of "include" to expose every function in that script? Thanks again, -V On Tue, Feb 10, 2015 at 7:16 PM, Laura Creighton wrote: > In a message of Wed, 11 Feb 2015 01:06:00 +0100, Laura Creighton writes: > >In a message of Tue, 10 Feb 2015 15:38:02 -0800, vlyamtsev at gmail.com > writes: > >>I defined function Fatalln in "mydef.py" and it works fine if i call it > from "mydef.py", but when i try to call it from "test.py" in the same > folder: > >>import mydef > >>... > >>Fatalln "my test" > >>i have NameError: name 'Fatalln' is not defined > >>I also tried include('mydef.py') with the same result... > >>What is the right syntax? > >>Thanks > >>-- > >>https://mail.python.org/mailman/listinfo/python-list > > > >from mydef import Fatalln > > > > Also, please be warned. If you use a unix system, or a linux > system. There are lots of problems you can get into if you > expect something named 'test' to run your code. Because they > already have one in their shell, and that one wins, and so ... > well, test.py is safe. But if you rename it as a script and call > it the binary file test ... > > Bad and unexpected things happen. > > Name it 'testme' or something like that. Never have that problem again. > :) > > Been there, done that! > Laura > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Wed Feb 11 08:29:22 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 05:29:22 -0800 (PST) Subject: idle feature request In-Reply-To: References: Message-ID: <7be0d2e3-cf0f-4ba4-b995-0b17d3e0d877@googlegroups.com> On Wednesday, February 11, 2015 at 6:50:35 PM UTC+5:30, Fabien wrote: > On 11.02.2015 14:11, Rustom Mody wrote: > > Context: > > I am using idle for taking python classes. > > I know this is not your question, but: have you considered using iPython > notebooks for teaching? They save automatically, look nice, and can be > sent as code, html, rst, ... > > Fabien Yeah I have considered (and thanks for reminding). However we are at that peculiar point where - de jure college provides computers - de facto everyone has a laptop Exploiting the second is a big advantage but its also chaos. Some windows; some linux; some macs etc So getting up and running with minimal headache is an important priority. From fabien.maussion at gmail.com Wed Feb 11 08:48:41 2015 From: fabien.maussion at gmail.com (Fabien) Date: Wed, 11 Feb 2015 14:48:41 +0100 Subject: idle feature request References: <7be0d2e3-cf0f-4ba4-b995-0b17d3e0d877@googlegroups.com> Message-ID: On 11.02.2015 14:29, Rustom Mody wrote: > So getting up and running with minimal headache is an important priority. Yes true. I consider Notebooks as a way to do "frontal" teaching, not necessarily usefull if students have to redo what you are doing on screen. For that, notebooks are rather prone to confusion because the order with wich you run the cells is important, so total beginners are really confused about variables overwritting etc. Fabien From clarence1126 at gmail.com Wed Feb 11 09:28:20 2015 From: clarence1126 at gmail.com (Clarence) Date: Wed, 11 Feb 2015 06:28:20 -0800 (PST) Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: <3f48cb2f-0e2a-4f47-a4cf-de658fb7b4f0@googlegroups.com> On Wednesday, February 11, 2015 at 8:04:33 AM UTC-5, Albert van der Horst wrote: > In article , > Ethan Furman wrote: > >-=-=-=-=-=- > > > >On 01/24/2015 11:55 AM, Chris Angelico wrote: > >> On Sun, Jan 25, 2015 at 5:56 AM, Ethan Furman wrote: > >>> If the non-generic is what you're concerned about: > >>> > >>> # not tested > >>> dispatch_table_a = {} > >>> dispatch_table_b = {} > >>> dispatch_table_c = {} > >>> > >>> class dispatch: > >>> def __init__(self, dispatch_table): > >>> self.dispatch = dispatch_table > >>> def __call__(self, func): > >>> self.dispatch[func.__name__] = func > >>> return func > >>> > >>> @dispatch(dispatch_table_a) > >>> def foo(...): > >>> pass > >> > >> That's still only able to assign to a key of a dictionary, using the > >> function name. > > > >This is a Good Thing. The def statement populates a few items, __name__ being one of them. One > >of the reasons lambda > >is not encouraged is because its name is always '', which just ain't helpful when the > >smelly becomes air borne! ;) > > That's the reason why my ideal Python doesn't attach a name to a lambda denotation: > > x -> x**2 > > is a function object, not something that has a name. > > It is not until we assign the object to a name (which becomes thereby a function) > that the __name__ thingy comes into play, like so. > > f = x->x**2 > or > f = x-> return x**2 > for those who don't like Algol68 > > I've heard arguments that with -> the __name__ is not filled in correctly. > I can't see why the parser would understand more easily > > def f(x): > return x**2 > > than > > f = x-> > return x**2 > > [I'm striving for simplification, doing away with both the lambda and > the def keywords. This is not a proposal for a language change, I'm > trying to explore possibilities here. ] > > > > >-- > >~Ethan~ > > Groetjes Albert > -- > Albert van der Horst, UTRECHT,THE NETHERLANDS > Economic growth -- being exponential -- ultimately falters. > albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst And then you do g = f And what then? Or even g = f = x -> x ** 2 Of course, you can say that problem already exists def f(x): return x ** 2 g = f But that goes to the heart of the issue: part of the purpose of def is to give a function a name. From d at davea.name Wed Feb 11 10:07:06 2015 From: d at davea.name (Dave Angel) Date: Wed, 11 Feb 2015 10:07:06 -0500 Subject: function inclusion problem In-Reply-To: References: <201502110006.t1B060VZ021974@fido.openend.se> <201502110016.t1B0GVah024279@fido.openend.se> Message-ID: <54DB701A.9060203@davea.name> On 02/11/2015 08:27 AM, Victor L wrote: > Laura, thanks for the answer - it works. Is there some equivalent of > "include" to expose every function in that script? > Thanks again, > -V > Please don't top-post, and please use text email, not html. Thank you. yes, as sohcahtoa82 at gmail.com pointed out, you can do from mydef import * But this is nearly always bad practice. If there are only a few functions you need access to, you should do from mydef import Fatalln, func2, func42 and if there are tons of them, you do NOT want to pollute your local namespace with them, and should do: import mydef x = mydef.func2() # or whatever The assumption is that when the code is in an importable module, it'll be maintained somewhat independently of the calling script/module. For example, if you're using a third party library, it could be updated without your having to rewrite your own calling code. So what happens if the 3rd party adds a new function, and you happen to have one by the same name. If you used the import* semantics, you could suddenly have broken code, with the complaint "But I didn't change a thing." Similarly, if you import from more than one module, and use the import* form, they could conflict with each other. And the order of importing will (usually) determine which names override which ones. The time that it's reasonable to use import* is when the third-party library already recommends it. They should only do so if they have written their library to only expose a careful subset of the names declared, and documented all of them. And when they make new releases, they're careful to hide any new symbols unless carefully documented in the release notes, so you can manually check for interference. Incidentally, this is also true of the standard library. There are symbols that are importable from multiple places, and sometimes they have the same meanings, sometimes they don't. An example (in Python 2.7) of the latter is os.walk and os.path.walk When I want to use one of those functions, I spell it out: for dirpath, dirnames, filenames in os.walk(topname): That way, there's no doubt in the reader's mind which one I intended. -- DaveA From python.list at tim.thechases.com Wed Feb 11 10:22:10 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 11 Feb 2015 09:22:10 -0600 Subject: function inclusion problem In-Reply-To: <54DB701A.9060203@davea.name> References: <201502110006.t1B060VZ021974@fido.openend.se> <201502110016.t1B0GVah024279@fido.openend.se> <54DB701A.9060203@davea.name> Message-ID: <20150211092210.38738d3f@bigbox.christie.dr> On 2015-02-11 10:07, Dave Angel wrote: > if there are tons of them, you do NOT want to pollute your local > namespace with them, and should do: > > import mydef > > x = mydef.func2() # or whatever or, if that's verbose, you can give a shorter alias: import Tkinter as tk root = tk.Tk() root.mainloop() -tkc From rosuav at gmail.com Wed Feb 11 10:37:16 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Feb 2015 02:37:16 +1100 Subject: function inclusion problem In-Reply-To: <54DB701A.9060203@davea.name> References: <201502110006.t1B060VZ021974@fido.openend.se> <201502110016.t1B0GVah024279@fido.openend.se> <54DB701A.9060203@davea.name> Message-ID: On Thu, Feb 12, 2015 at 2:07 AM, Dave Angel wrote: > Similarly, if you import from more than one module, and use the import* > form, they could conflict with each other. And the order of importing will > (usually) determine which names override which ones. Never mind about conflicts and order of importing... just try figuring out code like this: from os import * from sys import * from math import * # Calculate the total size of all files in a directory tot = 0 for path, dirs, files in walk(argv[1]): # We don't need to sum the directories separately for f in files: # getsizeof() returns a value in bytes tot += getsizeof(f)/1024.0/1024.0 print("Total directory size:", floor(tot), "MB") Now, I'm sure some of the experienced Python programmers here can see exactly what's wrong. But can everyone? I doubt it. Even if you run it, I doubt you'd get any better clue. But if you could see which module everything was imported from, it'd be pretty obvious. ChrisA From robert.kern at gmail.com Wed Feb 11 10:42:27 2015 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 11 Feb 2015 15:42:27 +0000 Subject: line_profiler: what am I doing wrong? In-Reply-To: <54daada4$0$12998$c3e8da3$5496439d@news.astraweb.com> References: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> <54daada4$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2015-02-11 01:17, Steven D'Aprano wrote: > Neal Becker wrote: > > >> To quote from https://pypi.python.org/pypi/line_profiler/ >> >> $ kernprof -l script_to_profile.py >> kernprof will create an instance of LineProfiler and insert it into the >> __builtins__ namespace with the name profile. > > Ewww!!!! What a Ruby-esque interface, that makes me sad :-( This is not a production library. It's a development tool designed to help developers shorten the cycle time for investigating these kinds of issues. Well, *a* developer; i.e. me. If it helps anyone else, yahtzee! > And what if you > have your own profile global name? Then you can pull it out from __builtin__ with a different name and use that other name. > And *wrong* too. `__builtins__` is a private CPython implementation detail. > The way to monkey-patch the built-ins in Python 2 is to inject the object > into `__builtin__` (no s), or `builtins` in Python 3. And indeed that is how it is implemented. Referring to that namespace as the "`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking I've implemented it one particular way, if you are desperate to find a nit to pick. > Seeing as > line_profiler is written in C, perhaps the author (Robert Kern) doesn't > care about supporting Jython or IronPython, but there may be Python > implementations (PyPy perhaps?) which can run C code but don't have > __builtins__. Indeed, I do not care about any of them. PyPy does not implement CPython's tracing API: https://bitbucket.org/pypy/pypy/src/2b2163d65ee437646194a1ceb2a3153db24c5f7e/pypy/module/cpyext/stubs.py?at=default#cl-1286 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Wed Feb 11 10:44:54 2015 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 11 Feb 2015 15:44:54 +0000 Subject: line_profiler: what am I doing wrong? In-Reply-To: References: Message-ID: On 2015-02-11 00:06, Neal Becker wrote: > I inserted > @profile > def run(...) > > into a module-level global function called 'run'. Something is very wrong here. > 1. profile results were written before anything even ran > 2. profile is not defined? > > kernprof -l ./test_unframed.py --lots --of --args ... > > Wrote profile results to test_unframed.py.lprof > Traceback (most recent call last): > File "/home/nbecker/.local/bin/kernprof", line 9, in > load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() > File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, > in main > execfile(script_file, ns, ns) > File "./test_unframed.py", line 721, in > @profile > NameError: name 'profile' is not defined Can you pare this down to a minimal complete example that fails in this way? Does a trivial module work? I.e. """ @profile def run(): pass run() """ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From nazimorganstanleyuccide at yahoo.com Wed Feb 11 11:27:25 2015 From: nazimorganstanleyuccide at yahoo.com (GIONA. FREEMASONS LONDON.) Date: Wed, 11 Feb 2015 08:27:25 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From ethan at stoneleaf.us Wed Feb 11 11:52:44 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 11 Feb 2015 08:52:44 -0800 Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: <54DB88DC.5090605@stoneleaf.us> On 02/11/2015 05:04 AM, Albert van der Horst wrote: > That's the reason why my ideal Python doesn't attach a name to a lambda denotation: > > x -> x**2 Hmmm. So now let's say you have a few dozen of these types of functions, and you want to talk about the twenty-sixth one with a peer... are you going to say, "Hey, look at the twenty-sixth one -- I think it has a bug." or something more like, "Hey, I think there's a bug in the hyp sin function." ? People use names, names are good. Python uses names, because names are good. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From marko at pacujo.net Wed Feb 11 12:12:46 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Feb 2015 19:12:46 +0200 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> Message-ID: <87twyswewh.fsf@elektro.pacujo.net> Rustom Mody : > On Wednesday, February 11, 2015 at 6:06:03 PM UTC+5:30, Marko Rauhamaa wrote: > >> The problem with xkbmap is that I don't know how to specify a new >> keyboard map as a regular user. I know how to do that with .Xmodmap. > > Xmodmap is neat... but obsolete Still, if xkbmap can't do what I need, I'm left with no option. Reading up on it at I used xkbcomp to dump my current keyboard definition. It produced a somewhat cryptic .xkb file of 1904 lines! My .Xmodmap, which remaps almost every single key on my keyboard contains 55 lines. The document has a funny view to keyboard layout: Consider making it a service launching after X starts, since reloaded configurations do not survive a system reboot. A regular user usually doesn't control system services. Plus, every user will likely have their personal keyboard layout preferences. Marko From skip.montanaro at gmail.com Wed Feb 11 12:23:10 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 11 Feb 2015 11:23:10 -0600 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <87twyswewh.fsf@elektro.pacujo.net> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> Message-ID: Thanks for all the ideas. As I'm an Emacs user (since Gosmacs in the early 80s), I will likely focus my attention there first. While the xkbmap/Xmodmap path seems like it would also work on Linux, I'm guessing Apple wouldn't allow that sort of thing to sully Mac OS X... Despite its name Skip From marko at pacujo.net Wed Feb 11 12:27:08 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Feb 2015 19:27:08 +0200 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> Message-ID: <87mw4kwe8j.fsf@elektro.pacujo.net> Skip Montanaro : > Thanks for all the ideas. As I'm an Emacs user (since Gosmacs in the > early 80s), I will likely focus my attention there first. While the > xkbmap/Xmodmap path seems like it would also work on Linux, I'm > guessing Apple wouldn't allow that sort of thing to sully Mac OS X... > Despite its name Oh? Marko From rustompmody at gmail.com Wed Feb 11 12:34:14 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 09:34:14 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <87twyswewh.fsf@elektro.pacujo.net> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> Message-ID: <7a6d0dfd-8850-4740-8981-f9ea190b955a@googlegroups.com> On Wednesday, February 11, 2015 at 10:42:59 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > > > On Wednesday, February 11, 2015 at 6:06:03 PM UTC+5:30, Marko Rauhamaa wrote: > > > >> The problem with xkbmap is that I don't know how to specify a new > >> keyboard map as a regular user. I know how to do that with .Xmodmap. > > > > Xmodmap is neat... but obsolete > > Still, if xkbmap can't do what I need, I'm left with no option. xkb supercedes xmodmap which is increasingly unsupported ie it does all of the earlier functionality and much more Only catch is to have to read impenetrable docs :-) Ive been directed to these http://pascal.tsu.ru/en/xkb/setup.html http://www.charvolant.org/~doug/xkb/html/index.html If you make some headway with these I'd like to know! From marko at pacujo.net Wed Feb 11 12:54:19 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Feb 2015 19:54:19 +0200 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> <7a6d0dfd-8850-4740-8981-f9ea190b955a@googlegroups.com> Message-ID: <874mqswcz8.fsf@elektro.pacujo.net> Rustom Mody : > xkb supercedes xmodmap which is increasingly unsupported > ie it does all of the earlier functionality and much more > > Only catch is to have to read impenetrable docs :-) > > Ive been directed to these > http://pascal.tsu.ru/en/xkb/setup.html > http://www.charvolant.org/~doug/xkb/html/index.html > > If you make some headway with these I'd like to know! Last I checked all I could find was written in Russian, which to my regret, I can't read. I think xmodmap will be there until Wayland replaces X11. Anyway, again, shows the utterly wrong approach to configuring the keyboard: you are supposed to touch the XF86Config-4 file. Keyboard preferences should be defined under $HOME with user preferences. It appears X.org lost sight of the fact that Unix is a multiuser environment where every user furnishes and decorates their $HOME to their liking. Marko From breamoreboy at yahoo.co.uk Wed Feb 11 13:00:01 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 11 Feb 2015 18:00:01 +0000 Subject: idle feature request In-Reply-To: References: Message-ID: On 11/02/2015 13:11, Rustom Mody wrote: > Context: > I am using idle for taking python classes. > > Finish the class and run out usually in a hurry and forget to save the > idle interaction window. Would like to save it so that I can mail it to the students. > > In emacs I could set a hook to make arbitrary 'buffers' like the python-idle > shell become 'saving-buffers' like typical file buffers. > ie in emacs like most other editors, a modified file is ask-to-save > The same can be setup for other (non-file) buffers like a python interaction shell > > Can I do the same for idle. > > [Note it would be great to have this feature in idle but I am only looking for > a hand in where in the idle sources I an fudge a patch for my own use > ] > If there isn't an enhancement request on the bug tracker for this please raise one, that way this doesn't get lost. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From lac at openend.se Wed Feb 11 13:01:43 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 11 Feb 2015 19:01:43 +0100 Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: Message from Skip Montanaro of "Wed, 11 Feb 2015 11:23:10 -0600." References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> Message-ID: <201502111801.t1BI1hD1018238@fido.openend.se> In a message of Wed, 11 Feb 2015 11:23:10 -0600, Skip Montanaro writes: >Thanks for all the ideas. As I'm an Emacs user (since Gosmacs in the >early 80s), I will likely focus my attention there first. While the >xkbmap/Xmodmap path seems like it would also work on Linux, I'm >guessing Apple wouldn't allow that sort of thing to sully Mac OS X... >Despite its name > >Skip >-- >https://mail.python.org/mailman/listinfo/python-list Looks to me as if this would work on your Mac. But I don't have one, so this is untested. http://lifehacker.com/5882684/the-best-keyboard-remapper-for-mac-os-x Laura From rustompmody at gmail.com Wed Feb 11 13:02:33 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 10:02:33 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: <874mqswcz8.fsf@elektro.pacujo.net> References: <93b272c6-ff67-4a7d-907e-f04606f1668c@googlegroups.com> <54dad63e$0$2852$c3e8da3$76491128@news.astraweb.com> <4614d7a2-c3b0-452d-9c68-068a5bc29c70@googlegroups.com> <874mqsbp7e.fsf@elektro.pacujo.net> <193d982c-3dfe-466c-bee0-8a1f6e2d1463@googlegroups.com> <87twyswewh.fsf@elektro.pacujo.net> <7a6d0dfd-8850-4740-8981-f9ea190b955a@googlegroups.com> <874mqswcz8.fsf@elektro.pacujo.net> Message-ID: On Wednesday, February 11, 2015 at 11:24:33 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody wrote: > > > xkb supercedes xmodmap which is increasingly unsupported > > ie it does all of the earlier functionality and much more > > > > Only catch is to have to read impenetrable docs :-) > > > > Ive been directed to these > > http://pascal.tsu.ru/en/xkb/setup.html > > http://www.charvolant.org/~doug/xkb/html/index.html > > > > If you make some headway with these I'd like to know! > > Last I checked all I could find was written in Russian, which to my > regret, I can't read. > > I think xmodmap will be there until Wayland replaces X11. Just found these https://lists.debian.org/debian-x/2012/09/msg00274.html (attachment) which points to http://madduck.net/docs/extending-xkb/ > > Anyway, again, > > > > shows the utterly wrong approach to configuring the keyboard: you are > supposed to touch the XF86Config-4 file. Keyboard preferences should be > defined under $HOME with user preferences. > > It appears X.org lost sight of the fact that Unix is a multiuser > environment where every user furnishes and decorates their $HOME to > their liking. All true. Doesn't change the fact that you are running out of time if you use xmodmap (see motivations of 1st link above) From hniksic at gmail.com Wed Feb 11 15:02:04 2015 From: hniksic at gmail.com (=?UTF-8?B?SHJ2b2plIE5pa8WhacSH?=) Date: Wed, 11 Feb 2015 21:02:04 +0100 Subject: Floating point "g" format not stripping trailing zeros Message-ID: According to the documentation of the "g" floating-point format, trailing zeros should be stripped from the resulting string: """ General format. For a given precision p >= 1, this rounds the number to p significant digits and then formats the result in either fixed-point format or in scientific notation, depending on its magnitude.[...] In both cases insignificant trailing zeros are removed from the significand, and the decimal point is also removed if there are no remaining digits following it. """ However, in some cases, the trailing zeros apparently remain: >>> from decimal import Decimal as D >>> x = D(1)/D(999) >>> '{:.15g}'.format(x) '0.00100100100100100' For floats, the trailing zeros are removed: >>> '{:.15g}'.format(1. / 999) '0.001001001001001' This behavior is present in both 2.7.8 and 3.4.1. Is this a bug in the formatting of Decimals? From johnny.t.shi at gmail.com Wed Feb 11 16:21:52 2015 From: johnny.t.shi at gmail.com (Johnny Ting Shi) Date: Wed, 11 Feb 2015 13:21:52 -0800 (PST) Subject: Is there a way to execute shell commands on VMWare server via PyVMomi? Message-ID: <8c0f89fc-81e0-4cdc-9824-e72fc0c7f756@googlegroups.com> We have some VM running on top of EXSi. We want to be to run some remote arbitrary commands on the VM, anyone has experience with https://github.com/vmware/pyvmomi? which command do i need to call? Thanks From breamoreboy at yahoo.co.uk Wed Feb 11 16:48:12 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 11 Feb 2015 21:48:12 +0000 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On 11/02/2015 20:02, Hrvoje Nik?i? wrote: > According to the documentation of the "g" floating-point format, > trailing zeros should be stripped from the resulting string: > > """ > General format. For a given precision p >= 1, this rounds the number > to p significant digits and then formats the result in either > fixed-point format or in scientific notation, depending on its > magnitude.[...] > In both cases insignificant trailing zeros are removed from the > significand, and the decimal point is also removed if there are no > remaining digits following it. > """ > > However, in some cases, the trailing zeros apparently remain: > >>>> from decimal import Decimal as D >>>> x = D(1)/D(999) >>>> '{:.15g}'.format(x) > '0.00100100100100100' > > For floats, the trailing zeros are removed: > >>>> '{:.15g}'.format(1. / 999) > '0.001001001001001' > > This behavior is present in both 2.7.8 and 3.4.1. Is this a bug in the > formatting of Decimals? > I'd say it's a bug. P is 15, you've got 17 digits after the decimal place and two of those are insignificant trailing zeros. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Feb 11 17:17:20 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 11 Feb 2015 22:17:20 +0000 Subject: warnings building 3.5 on Windows Message-ID: There are loads of "possible loss of data" warnings such as. 15>..\Modules\_tkinter.c(640): warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data 15>..\Modules\_tkinter.c(892): warning C4244: 'function' : conversion from 'Py_ssize_t' to 'int', possible loss of data I understand that these are usually silenced. Should I raise this on the issue tracker? There are also some deprecation warnings such as 26>..\Modules\overlapped.c(977): warning C4996: 'WSAStringToAddressA': Use WSAStringToAddressW() instead or define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings 26> C:\Program Files (x86)\Windows Kits\8.1\Include\um\winsock2.h(3623) : see declaration of 'WSAStringToAddressA' Should I raise this if only as a place holder? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Wed Feb 11 17:19:26 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 11 Feb 2015 15:19:26 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Wed, Feb 11, 2015 at 2:48 PM, Mark Lawrence wrote: > On 11/02/2015 20:02, Hrvoje Nik?i? wrote: >> >> According to the documentation of the "g" floating-point format, >> trailing zeros should be stripped from the resulting string: >> >> """ >> General format. For a given precision p >= 1, this rounds the number >> to p significant digits and then formats the result in either >> fixed-point format or in scientific notation, depending on its >> magnitude.[...] >> In both cases insignificant trailing zeros are removed from the >> significand, and the decimal point is also removed if there are no >> remaining digits following it. >> """ >> >> However, in some cases, the trailing zeros apparently remain: >> >>>>> from decimal import Decimal as D >>>>> x = D(1)/D(999) >>>>> '{:.15g}'.format(x) >> >> '0.00100100100100100' >> >> For floats, the trailing zeros are removed: >> >>>>> '{:.15g}'.format(1. / 999) >> >> '0.001001001001001' >> >> This behavior is present in both 2.7.8 and 3.4.1. Is this a bug in the >> formatting of Decimals? >> > > I'd say it's a bug. P is 15, you've got 17 digits after the decimal place > and two of those are insignificant trailing zeros. Actually it's the float version that doesn't match the documentation. In the decimal version, sure there are 17 digits after the decimal place there, but the first two -- which are leading zeroes -- would not normally be considered significant. The float version OTOH is only giving you 13 significant digits when 15 were requested. For illustration, compare the same formatting with the scale ramped up to make it switch to exponential notation: >>> '{:.15g}'.format(D(1) / D(999000000)) '1.00100100100100e-9' >>> '{:.15g}'.format(1./999000000) '1.001001001001e-09' From ian.g.kelly at gmail.com Wed Feb 11 17:28:33 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 11 Feb 2015 15:28:33 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Wed, Feb 11, 2015 at 3:19 PM, Ian Kelly wrote: > On Wed, Feb 11, 2015 at 2:48 PM, Mark Lawrence wrote: >> On 11/02/2015 20:02, Hrvoje Nik?i? wrote: >>> >>> According to the documentation of the "g" floating-point format, >>> trailing zeros should be stripped from the resulting string: >>> >>> """ >>> General format. For a given precision p >= 1, this rounds the number >>> to p significant digits and then formats the result in either >>> fixed-point format or in scientific notation, depending on its >>> magnitude.[...] >>> In both cases insignificant trailing zeros are removed from the >>> significand, and the decimal point is also removed if there are no >>> remaining digits following it. >>> """ >>> >>> However, in some cases, the trailing zeros apparently remain: >>> >>>>>> from decimal import Decimal as D >>>>>> x = D(1)/D(999) >>>>>> '{:.15g}'.format(x) >>> >>> '0.00100100100100100' >>> >>> For floats, the trailing zeros are removed: >>> >>>>>> '{:.15g}'.format(1. / 999) >>> >>> '0.001001001001001' >>> >>> This behavior is present in both 2.7.8 and 3.4.1. Is this a bug in the >>> formatting of Decimals? >>> >> >> I'd say it's a bug. P is 15, you've got 17 digits after the decimal place >> and two of those are insignificant trailing zeros. > > Actually it's the float version that doesn't match the documentation. > In the decimal version, sure there are 17 digits after the decimal > place there, but the first two -- which are leading zeroes -- would > not normally be considered significant. The float version OTOH is only > giving you 13 significant digits when 15 were requested. > > For illustration, compare the same formatting with the scale ramped up > to make it switch to exponential notation: > >>>> '{:.15g}'.format(D(1) / D(999000000)) > '1.00100100100100e-9' >>>> '{:.15g}'.format(1./999000000) > '1.001001001001e-09' Although here's something else that is definitely an issue with the decimal formatting: >>> '{:.15g}'.format(D(1)/D(999000)) '0.00000100100100100100' This has an exponent of -6, so according to the docs it should be formatted in exponential notation. From rosuav at gmail.com Wed Feb 11 17:31:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Feb 2015 09:31:44 +1100 Subject: warnings building 3.5 on Windows In-Reply-To: References: Message-ID: On Thu, Feb 12, 2015 at 9:17 AM, Mark Lawrence wrote: > 26> C:\Program Files (x86)\Windows > Kits\8.1\Include\um\winsock2.h(3623) : see declaration of > 'WSAStringToAddressA' To clarify, are you building a 32-bit or 64-bit Python here? Also, what version of Windows? It looks like at least some of the buildbots are showing similar warnings, eg: http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/411/steps/compile/logs/warnings%20%28396%29 Which would imply that, yes, it's probably best to raise that as an issue. ChrisA From mail at timgolden.me.uk Wed Feb 11 17:38:33 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 11 Feb 2015 22:38:33 +0000 Subject: warnings building 3.5 on Windows In-Reply-To: References: Message-ID: <54DBD9E9.3070007@timgolden.me.uk> On 11/02/2015 22:31, Chris Angelico wrote: > On Thu, Feb 12, 2015 at 9:17 AM, Mark Lawrence wrote: >> 26> C:\Program Files (x86)\Windows >> Kits\8.1\Include\um\winsock2.h(3623) : see declaration of >> 'WSAStringToAddressA' > > To clarify, are you building a 32-bit or 64-bit Python here? Also, > what version of Windows? > > It looks like at least some of the buildbots are showing similar warnings, eg: > http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/411/steps/compile/logs/warnings%20%28396%29 > > Which would imply that, yes, it's probably best to raise that as an issue. > > ChrisA > Agreed. Please log a ticket; if you set the component to "Windows" then Steve, Zach & I will all be nosied. Definitely specify whether you're building 32 or 64-bit. And which compiler you're using. TJG From steve+comp.lang.python at pearwood.info Wed Feb 11 17:42:40 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 12 Feb 2015 09:42:40 +1100 Subject: Is there a way to execute shell commands on VMWare server via PyVMomi? References: <8c0f89fc-81e0-4cdc-9824-e72fc0c7f756@googlegroups.com> Message-ID: <54dbdae1$0$12987$c3e8da3$5496439d@news.astraweb.com> Johnny Ting Shi wrote: > We have some VM running on top of EXSi. We want to be to run some remote > arbitrary commands on the VM, anyone has experience with > https://github.com/vmware/pyvmomi? which command do i need to call? I'm afraid I've never used pyvmomi, but I'm not sure why the fact it is a VM is relevant. You want to run commands on another machine (hence, remote commands), whether that machine is a physical machine or a VM shouldn't matter. Is there something special about the host or VM that prevents normal remote procedure calls from working? Do you mean OS commands? That suggests using ssh on Linux machines. If you want to run Python code remotely, I have used both Pyro and Rpyc, they are both very mature and quite good. https://pypi.python.org/pypi/rpyc/ https://pypi.python.org/pypi/Pyro4/ Watch out though, there is another project calling itself "Pyro" too, Python Robotics. I don't know which has seniority. -- Steven From steve+comp.lang.python at pearwood.info Wed Feb 11 18:02:15 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 12 Feb 2015 10:02:15 +1100 Subject: line_profiler: what am I doing wrong? References: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> <54daada4$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54dbdf78$0$13004$c3e8da3$5496439d@news.astraweb.com> Robert Kern wrote: > Referring to that namespace as the > "`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking > I've implemented it one particular way Well it certainly mislead me :-) -- Steven From breamoreboy at yahoo.co.uk Wed Feb 11 18:39:12 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 11 Feb 2015 23:39:12 +0000 Subject: warnings building 3.5 on Windows In-Reply-To: <54DBD9E9.3070007@timgolden.me.uk> References: <54DBD9E9.3070007@timgolden.me.uk> Message-ID: On 11/02/2015 22:38, Tim Golden wrote: > On 11/02/2015 22:31, Chris Angelico wrote: >> On Thu, Feb 12, 2015 at 9:17 AM, Mark Lawrence >> wrote: >>> 26> C:\Program Files (x86)\Windows >>> Kits\8.1\Include\um\winsock2.h(3623) : see declaration of >>> 'WSAStringToAddressA' >> >> To clarify, are you building a 32-bit or 64-bit Python here? Also, >> what version of Windows? >> >> It looks like at least some of the buildbots are showing similar >> warnings, eg: >> http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/411/steps/compile/logs/warnings%20%28396%29 >> >> >> Which would imply that, yes, it's probably best to raise that as an >> issue. >> >> ChrisA >> > > Agreed. Please log a ticket; if you set the component to "Windows" then > Steve, Zach & I will all be nosied. Definitely specify whether you're > building 32 or 64-bit. And which compiler you're using. > > TJG http://bugs.python.org/issue23449 fatal error building tkinter http://bugs.python.org/issue23450 possible loss of data http://bugs.python.org/issue23449 deprecation warnings. Now I'll go and add the compiler. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From nad at acm.org Wed Feb 11 19:23:05 2015 From: nad at acm.org (Ned Deily) Date: Wed, 11 Feb 2015 16:23:05 -0800 Subject: building c extensions with setuptools that are not tied to python installed on build machine References: Message-ID: In article , Matthew Taylor wrote: > Does this make sense to anyone? I'm still a little new to Python in > general (especially binary packaging), and it seems like this would be > a common problem for any projects with C extensions that need broad > binary distribution. Does anyone have any suggestions? Please take a > look at our setup.py file [4] and tell me if we're doing something > egregiously wrong. Just taking a quick look at your setup.py there shows a quite complicated build, including SWIG. One suggestion: keep in mind that it's normal on OS X for the absolute path of shared libraries and frameworks to be embedded in the linked binaries, including the C (or C++) extension module bundles (.so files) built for Python packages. If any of the .so files or any other binary artifacts (executables, shared libraries) created by your package are linked to the Python interpreter's shared library, that may be why you are getting a mixture of Python instances. One way to check for this is to use: otool -L *.so *.dylib on all of the directories containing linked binaries in your project and check for paths like: /System/Library/Frameworks/Python.framework That would be a link to the Apple-supplied system Python. A link to /Library/Frameworks/Python.framework or some other path would be to a third-party Python like from python.org or Homebrew. Due to differences in how the various Pythons are built and depending on what the C or C++ code is doing, it may not be possible to have one binary wheel that works with different Python instances of the same version. For many simple projects, it does work. You *could* also ask on the PythonMac SIG list. https://mail.python.org/mailman/listinfo/pythonmac-sig -- Ned Deily, nad at acm.org From matt at numenta.org Wed Feb 11 20:44:48 2015 From: matt at numenta.org (Matthew Taylor) Date: Wed, 11 Feb 2015 17:44:48 -0800 Subject: building c extensions with setuptools that are not tied to python installed on build machine In-Reply-To: References: Message-ID: Ned, thank you for your insight on this problem. I will take your advice and do some more digging. You've been very helpful. Regards, --------- Matt Taylor OS Community Flag-Bearer Numenta On Wed, Feb 11, 2015 at 4:23 PM, Ned Deily wrote: > In article > , > Matthew Taylor wrote: >> Does this make sense to anyone? I'm still a little new to Python in >> general (especially binary packaging), and it seems like this would be >> a common problem for any projects with C extensions that need broad >> binary distribution. Does anyone have any suggestions? Please take a >> look at our setup.py file [4] and tell me if we're doing something >> egregiously wrong. > > Just taking a quick look at your setup.py there shows a quite > complicated build, including SWIG. One suggestion: keep in mind that > it's normal on OS X for the absolute path of shared libraries and > frameworks to be embedded in the linked binaries, including the C (or > C++) extension module bundles (.so files) built for Python packages. If > any of the .so files or any other binary artifacts (executables, shared > libraries) created by your package are linked to the Python > interpreter's shared library, that may be why you are getting a mixture > of Python instances. One way to check for this is to use: > > otool -L *.so *.dylib > > on all of the directories containing linked binaries in your project and > check for paths like: > /System/Library/Frameworks/Python.framework > > That would be a link to the Apple-supplied system Python. > > A link to /Library/Frameworks/Python.framework or some other path would > be to a third-party Python like from python.org or Homebrew. > > Due to differences in how the various Pythons are built and depending on > what the C or C++ code is doing, it may not be possible to have one > binary wheel that works with different Python instances of the same > version. For many simple projects, it does work. > > You *could* also ask on the PythonMac SIG list. > > https://mail.python.org/mailman/listinfo/pythonmac-sig > > -- > Ned Deily, > nad at acm.org > > -- > https://mail.python.org/mailman/listinfo/python-list From tjreedy at udel.edu Wed Feb 11 21:06:28 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Feb 2015 21:06:28 -0500 Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: On 2/11/2015 8:04 AM, Albert van der Horst wrote: > It is not until we assign the object to a name (which becomes thereby a function) > that the __name__ thingy comes into play, like so. But __name__ would not come into play. > f = x->x**2 This would mean to create an anonymous function object and then bind 'f' to that object in the current local namespace. It would be the same as the discouraged f = lambda x: x**2 > I've heard arguments that with -> the __name__ is not filled in correctly. Because local namespace name binding does not and should not mutate the object the name is bound to. > I can't see why the parser would understand more easily > > def f(x): > return x**2 > than > > f = x-> > return x**2 The parser parses both equally well. That is not the issue. -- Terry Jan Reedy From tjreedy at udel.edu Wed Feb 11 21:27:25 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Feb 2015 21:27:25 -0500 Subject: idle feature request In-Reply-To: References: Message-ID: On 2/11/2015 1:00 PM, Mark Lawrence wrote: > On 11/02/2015 13:11, Rustom Mody wrote: >> Context: >> I am using idle for taking python classes. Teaching or taking? >> Finish the class and run out usually in a hurry and forget to save the >> idle interaction window. Do you mean the shell window? >> Would like to save it so that I can mail it to the students. Then save it. What am I not understainding? Your problem scenario is not clear to me. See below for a possible description that I do understand. >> In emacs I could set a hook to make arbitrary 'buffers' like the >> python-idle >> shell become 'saving-buffers' like typical file buffers. >> ie in emacs like most other editors, a modified file is ask-to-save That is true in Idle as well. >> The same can be setup for other (non-file) buffers like a python >> interaction shell I have never used emacs so I do not understand your reference thereto. In Windows, the interactive console history cannot be saved except by awkwardly selecting and cutting. Idle does allow the shell contents to be saved. >> Can I do the same for idle. "Same what?" is the question. > If there isn't an enhancement request on the bug tracker for this please > raise one, that way this doesn't get lost. Let us clarify the request here first. If one saves the shell with 'save as', the filename is added to the title bar. If one does something more, the name in bracketed with *s to indicate the the memory buffer has been changed since it was last saved. This is the same as regular edit windows. However, when one quits, then, unlike a regular edit window, there is no ask-to-save warning. Is this what you want? This would seem reasonable and likely easy. (I would, however, not want the prompt when the buffer has not been saved -- or this could be a configuration option.) -- Terry Jan Reedy From tjreedy at udel.edu Wed Feb 11 21:46:34 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Feb 2015 21:46:34 -0500 Subject: warnings building 3.5 on Windows In-Reply-To: References: Message-ID: On 2/11/2015 5:17 PM, Mark Lawrence wrote: > There are loads of "possible loss of data" warnings such as. > > 15>..\Modules\_tkinter.c(640): warning C4267: '+=' : conversion from > 'size_t' to 'int', possible loss of data > 15>..\Modules\_tkinter.c(892): warning C4244: 'function' : conversion > from 'Py_ssize_t' to 'int', possible loss of data Are these new? All in tkinter.c? Is so, can you find the patch responsible (hg annotate?)? > I understand that these are usually silenced. Should I raise this on > the issue tracker? The warnings *are* annoying, but I have mostly not bothered to say anything since Viktor Stinner tends to eventually fix new ones and make them go away. But feel free to go ahead, and add him as nosy. Also Serhiy Storchaka if all/mostly tkinter.c > There are also some deprecation warnings such as > > 26>..\Modules\overlapped.c(977): warning C4996: 'WSAStringToAddressA': > Use WSAStringToAddressW() instead or define > _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings > 26> C:\Program Files (x86)\Windows > Kits\8.1\Include\um\winsock2.h(3623) : see declaration of > 'WSAStringToAddressA' > > Should I raise this if only as a place holder? > -- Terry Jan Reedy From rustompmody at gmail.com Wed Feb 11 21:48:14 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 11 Feb 2015 18:48:14 -0800 (PST) Subject: idle feature request In-Reply-To: References: Message-ID: <3bf81ef4-e022-4cd3-88d5-637fbf532a88@googlegroups.com> On Thursday, February 12, 2015 at 7:57:48 AM UTC+5:30, Terry Reedy wrote: > On 2/11/2015 1:00 PM, Mark Lawrence wrote: > > On 11/02/2015 13:11, Rustom Mody wrote: > >> Context: > >> I am using idle for taking python classes. > > Teaching or taking? Teaching -- I would like to mail? students the interaction so they can pay more attention and less bother about taking notes. > > >> Finish the class and run out usually in a hurry and forget to save the > >> idle interaction window. > > Do you mean the shell window? Yeah > > >> Would like to save it so that I can mail it to the students. > > Then save it. What am I not understainding? Your problem scenario is > not clear to me. See below for a possible description that I do understand. > > >> In emacs I could set a hook to make arbitrary 'buffers' like the > >> python-idle > >> shell become 'saving-buffers' like typical file buffers. > >> ie in emacs like most other editors, a modified file is ask-to-save > > That is true in Idle as well. > > >> The same can be setup for other (non-file) buffers like a python > >> interaction shell > > I have never used emacs so I do not understand your reference thereto. > In Windows, the interactive console history cannot be saved except by > awkwardly selecting and cutting. Idle does allow the shell contents to > be saved. > > >> Can I do the same for idle. > > "Same what?" is the question. > > > If there isn't an enhancement request on the bug tracker for this please > > raise one, that way this doesn't get lost. > > Let us clarify the request here first. > > If one saves the shell with 'save as', the filename is added to the > title bar. If one does something more, the name in bracketed with *s to > indicate the the memory buffer has been changed since it was last saved. > This is the same as regular edit windows. > > However, when one quits, then, unlike a regular edit window, there is no > ask-to-save warning. Is this what you want? This would seem reasonable > and likely easy. (I would, however, not want the prompt when the buffer > has not been saved -- or this could be a configuration option.) Yeah thats what I want. In a file-window there is 'ask-to-save' as soon as a single character is typed. I want similar for shell window. As I said if there is a config option for that, thats best. But if I have to put in a few lines in my own idle that's fine too?. [BTW I have some other feature requests too. Is this the best forum to talk of them? And thanks for your work on idle :-) ] ? I just discovered that mail is passe and they would prefer whatsapp! Well whatever :-) ? I have a class tomorrow! (but I'll try to remember)!! From ian.g.kelly at gmail.com Wed Feb 11 21:57:20 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 11 Feb 2015 19:57:20 -0700 Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: On Wed, Feb 11, 2015 at 7:06 PM, Terry Reedy wrote: >> I can't see why the parser would understand more easily >> >> def f(x): >> return x**2 >> than >> >> f = x-> >> return x**2 > > > The parser parses both equally well. That is not the issue. The compiler could at some point recognize that the function is being assigned to a simple name and transform the assignment into a def for purposes of byte code generation. It could also do the same with lambda, although it currently doesn't. The reason I don't like this replacing def isn't because the name is necessarily lost. It's because the lack of the well-defined def statement encourages more complex usages like functions['f'] = x -> x**2 where such implicit transformations won't work. From rosuav at gmail.com Wed Feb 11 22:05:24 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Feb 2015 14:05:24 +1100 Subject: idle feature request In-Reply-To: References: Message-ID: On Thu, Feb 12, 2015 at 1:27 PM, Terry Reedy wrote: > However, when one quits, then, unlike a regular edit window, there is no > ask-to-save warning. Is this what you want? This would seem reasonable and > likely easy. (I would, however, not want the prompt when the buffer has not > been saved -- or this could be a configuration option.) That's my understanding of the request, too. It seems reasonable to have a way to be prompted about saving. ChrisA From rosuav at gmail.com Wed Feb 11 22:56:12 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Feb 2015 14:56:12 +1100 Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: On Thu, Feb 12, 2015 at 1:57 PM, Ian Kelly wrote: > The reason I don't like this replacing def isn't because the name is > necessarily lost. It's because the lack of the well-defined def > statement encourages more complex usages like > > functions['f'] = x -> x**2 > > where such implicit transformations won't work. That's actually where this started. I mean, if a function can be called "", why can't it be called "functions['f']"? ChrisA From breamoreboy at yahoo.co.uk Wed Feb 11 23:48:29 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 12 Feb 2015 04:48:29 +0000 Subject: idle feature request In-Reply-To: <3bf81ef4-e022-4cd3-88d5-637fbf532a88@googlegroups.com> References: <3bf81ef4-e022-4cd3-88d5-637fbf532a88@googlegroups.com> Message-ID: On 12/02/2015 02:48, Rustom Mody wrote: > > [BTW I have some other feature requests too. Is this the best forum to talk of them? And thanks for your work on idle :-) ] > Here is as good a place as any although there is an IDLE development mailing list at https://mail.python.org/mailman/listinfo/idle-dev or gmane.comp.python.idle Have you checked on the bug tracker to see what has already been requested? Also have you seen IDLEX http://idlex.sourceforge.net/ ? You might be able to beg, steal or borrow something from it and it's available from pypi. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From lk.palao at gmail.com Thu Feb 12 00:02:20 2015 From: lk.palao at gmail.com (Leo Kris Palao) Date: Thu, 12 Feb 2015 13:02:20 +0800 Subject: GDAL installation In-Reply-To: References: Message-ID: Hi Asim, thanks for your help. It is working properly now. Thanks, -Leo On Wed, Feb 11, 2015 at 4:48 PM, Asim Jalis wrote: > Hi Leo, > > This might be a PATH issue. > > See this discussion for details. > > > https://pythongisandstuff.wordpress.com/2011/07/07/installing-gdal-and-ogr-for-python-on-windows/ > > Asim > > On Tue, Feb 10, 2015 at 9:11 PM, Leo Kris Palao > wrote: > >> Hi Python Users, >> >> I currently installed the Python 2.7.9 and installed the GDAL package. >> First, I tried to install GDAL using PIP but it throws an error - I cannot >> remember the exact error message. So, I install it using easy_install >> command. But when I import the package I am getting this message, which I >> really don't understand. >> >> Microsoft Windows [Version 6.1.7601] >> Copyright (c) 2009 Microsoft Corporation. All rights reserved. >> >> C:\Users\lpalao>python >>> Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit >>> (AMD64)] on win32 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> import gdal >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "C:\Python27\Python2.7.9\lib\site-packages\gdal.py", line 2, in >>> >>> from osgeo.gdal import deprecation_warn >>> File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", >>> line 21, in >>> _gdal = swig_import_helper() >>> File "C:\Python27\Python2.7.9\lib\site-packages\osgeo\__init__.py", >>> line 17, in swig_import_helper >>> _mod = imp.load_module('_gdal', fp, pathname, description) >>> ImportError: DLL load failed: The specified module could not be found. >>> >>> >> >> >> Thanks in advance, >> -Leo >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dieter at handshake.de Thu Feb 12 01:48:06 2015 From: dieter at handshake.de (dieter) Date: Thu, 12 Feb 2015 07:48:06 +0100 Subject: urllib.parser.quote() and RFC 2396: unreserved characters get encoded References: Message-ID: <87y4o3wrq1.fsf@handshake.de> Bruno Cauet writes: > Unicode characters outside the ASCII range also get encoded when they > have no reason to, e.g. > >>> pathlib.PurePath("/home/??/").as_uri() > 'file:///home/%EC%8B%B8%EC%9D%B4' Non-ASCII characters are not legal uri characters. Look at section 2.3 of "http://www.faqs.org/rfcs/rfc2396.html". You see there "unreserved = alphanum | mark" with with "alphanum" defined in section 1.6 as the ASCII letters and digits. See also section 2.1 ("URI and non-ASCII characters"). It tells that non-ASCII characters should be utf-8 encoded and then uri-escaped. Thus, the handling (by "urllib") of non-ASCII unicode characters seems to be correct. From tjreedy at udel.edu Thu Feb 12 02:17:14 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Feb 2015 02:17:14 -0500 Subject: idle feature request In-Reply-To: <3bf81ef4-e022-4cd3-88d5-637fbf532a88@googlegroups.com> References: <3bf81ef4-e022-4cd3-88d5-637fbf532a88@googlegroups.com> Message-ID: On 2/11/2015 9:48 PM, Rustom Mody wrote: > On Thursday, February 12, 2015 at 7:57:48 AM UTC+5:30, Terry Reedy wrote: >> If one saves the shell with 'save as', the filename is added to the >> title bar. If one does something more, the name in bracketed with *s to >> indicate the the memory buffer has been changed since it was last saved. >> This is the same as regular edit windows. >> >> However, when one quits, then, unlike a regular edit window, there is no >> ask-to-save warning. Is this what you want? This would seem reasonable >> and likely easy. The Shell window is a subclass of the (base) Editor window. So enhancement 1, invoke ask-to-save after save-as should be easy. Find the current code (grep the message) and determine why it does not work for the shell. Saving the shell is, I believe, rare, and I believe there is half a chance that a person who has saved the shell once will want to do so again. >> (I would, however, not want the prompt when the buffer >> has not been saved -- or this could be a configuration option.) Enhancement 2 would be adding a configuration option to invoke ask-to-save even without save-as. I am sure that this would be harder. > Yeah thats what I want. > In a file-window there is 'ask-to-save' as soon as a single character is typed. > I want similar for shell window. > As I said if there is a config option for that, thats best. > But if I have to put in a few lines in my own idle that's fine too?. > [BTW I have some other feature requests too. Is this the best forum to talk of them? And thanks for your work on idle :-) ] Idle-sig is pretty dead, so go ahead here. You do not have to search the tracker if you do not want to. I have a categorized list of all Idle issues on the tracker (with issue numbers) and will look up ideas against that list anyway. -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Thu Feb 12 03:46:39 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 12 Feb 2015 19:46:39 +1100 Subject: Python discussed in Nature Message-ID: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> "Nature", one of the world's premier science journals, has published an excellent article about programming in Python: http://www.nature.com/news/programming-pick-up-python-1.16833 -- Steve From breamoreboy at yahoo.co.uk Thu Feb 12 04:19:25 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 12 Feb 2015 09:19:25 +0000 Subject: Python discussed in Nature In-Reply-To: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 12/02/2015 08:46, Steven D'Aprano wrote: > "Nature", one of the world's premier science journals, has published an > excellent article about programming in Python: > > http://www.nature.com/news/programming-pick-up-python-1.16833 > Interesting. I'll leave someone more diplomatic than myself to reply to the comment at the end of the article. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From wxjmfauth at gmail.com Thu Feb 12 04:31:31 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 12 Feb 2015 01:31:31 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> Le jeudi 12 f?vrier 2015 09:46:52 UTC+1, Steven D'Aprano a ?crit : > "Nature", one of the world's premier science journals, has published an > excellent article about programming in Python: > > http://www.nature.com/news/programming-pick-up-python-1.16833 > > > -- > Steve D:\>c:\python35\python.exe Python 3.5.0a1 (v3.5.0a1:5d4b6a57d5fd, Feb 7 2015, 17:58:38) [MSC v.1900 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print('abcoe? EURO') D:\>echo 'abc?oe EURO' crash 'abc?oe EURO' crash D:\> From fabien.maussion at gmail.com Thu Feb 12 06:07:58 2015 From: fabien.maussion at gmail.com (Fabien) Date: Thu, 12 Feb 2015 12:07:58 +0100 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> Message-ID: On 12.02.2015 10:31, wxjmfauth at gmail.com wrote: [some OT stuffs about unicode] ... what a coincidence then that a huge majority of scientists (including me) dont care AT ALL about unicode. But since scientists are not paid to rewrite old code, the scientific world is still stuck to python 2. It's a pitty, given how easy it is to write py2/py3 compatible scientific tools. Thanks for the link to the article steven! Fabien (sorry for the OT & sorry for feeding the t....) From marko at pacujo.net Thu Feb 12 06:25:20 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Feb 2015 13:25:20 +0200 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> Message-ID: <87zj8jcqxr.fsf@elektro.pacujo.net> Fabien : > ... what a coincidence then that a huge majority of scientists > (including me) dont care AT ALL about unicode. You shouldn't, any more than you care about ASCII or 2's-complement encoding. Things should just work. > But since scientists are not paid to rewrite old code, the scientific > world is still stuck to python 2. It's a pitty, given how easy it is > to write py2/py3 compatible scientific tools. What's a pity is that Python3 chose to ignore the seamless transition path. It would have been nice, for example, to have all Python 3 code explicitly mark its dialect (a .py3 extension, a magic import or something) and then allow legacy Py2 code and Py3 code coexist the same way C and C++ can coexist. Marko From fabien.maussion at gmail.com Thu Feb 12 06:33:56 2015 From: fabien.maussion at gmail.com (Fabien) Date: Thu, 12 Feb 2015 12:33:56 +0100 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> Message-ID: On 12.02.2015 12:25, Marko Rauhamaa wrote: > Fabien: > >> >... what a coincidence then that a huge majority of scientists >> >(including me) dont care AT ALL about unicode. > You shouldn't, any more than you care about ASCII or 2's-complement > encoding. Things should just work. And they do! Since almost one year in writing scientific python code, not a single problem. I wouldnt even know about issues if i didnt read some posts here. >> >But since scientists are not paid to rewrite old code, the scientific >> >world is still stuck to python 2. It's a pitty, given how easy it is >> >to write py2/py3 compatible scientific tools. > What's a pity is that Python3 chose to ignore the seamless transition > path. It would have been nice, for example, to have all Python 3 code > explicitly mark its dialect (a .py3 extension, a magic import or > something) and then allow legacy Py2 code and Py3 code coexist the same > way C and C++ can coexist. But this was exactly my point! Today in 2015 it's incredibly easy to write py2/py3 code for a scientist. The whole SciPy track has done the transition. Not an issue anymore either, for me at least (python youngster ;-) Fabien From nomail at invalid.com Thu Feb 12 07:26:21 2015 From: nomail at invalid.com (ast) Date: Thu, 12 Feb 2015 13:26:21 +0100 Subject: Bad text appearance in IDLE Message-ID: <54dc9bee$0$3046$426a34cc@news.free.fr> Hello Here is how text appears in IDLE window http://www.cjoint.com/data/0BmnEIcxVAx.htm Yesterday evening I had not this trouble. It appears this morning. I restarted my computer with no effect. A windows Vista update has been done this morning, with about 10 fixes. I suspect something gone wrong with this update Has somebody an explanation about that ? thx From charles.sartori at gmail.com Thu Feb 12 07:34:03 2015 From: charles.sartori at gmail.com (charles.sartori at gmail.com) Date: Thu, 12 Feb 2015 04:34:03 -0800 (PST) Subject: Group by interval time Message-ID: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> Hello there! I`m trying to group by a list of Row() objects in 12days interval and sum(). values. Here is an example of the list [Row(time=datetime.datetime(2013, 1, 1, 0, 0), sum=4676557380615), Row(time=datetime.datetime(2013, 1, 2, 0, 0), sum=6549630855895), Row(time=datetime.datetime(2013, 1, 3, 0, 0), sum=6549630855895), ...] Row() objects has two vars: row.time and row.sum The result that I`m looking for is: [[datetime.datetime(2013, 1, 1, 0, 0), value], [datetime.datetime(2013, 1, 12, 0, 0), value], [datetime.datetime(2013, 1, 24, 0, 0), value] ] Where value is the sum() of all row.sum in that interval. I`m trying to use itertools.groupby by I could not get it to work yet. Thnak you. From lk.palao at gmail.com Thu Feb 12 07:51:28 2015 From: lk.palao at gmail.com (Leo Kris Palao) Date: Thu, 12 Feb 2015 20:51:28 +0800 Subject: Random forest and svm for remote sensing in python In-Reply-To: References: Message-ID: Hi Python Users, Good day! I am currently using ENVI for my image processing/remote sensing work, but would love to divert into open source python programming for remote sensing. Can you give me some good sites where I can see practical examples of how python is used for remote sensing specially using random forest and support vector machine algorithms. Thanks, -Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Feb 12 08:17:42 2015 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 12 Feb 2015 13:17:42 +0000 Subject: Group by interval time In-Reply-To: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> References: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> Message-ID: <54DCA7F6.3070903@mrabarnett.plus.com> On 2015-02-12 12:34, charles.sartori at gmail.com wrote: > Hello there! > > I`m trying to group by a list of Row() objects in 12days interval and sum(). values. Here is an example of the list > > [Row(time=datetime.datetime(2013, 1, 1, 0, 0), sum=4676557380615), Row(time=datetime.datetime(2013, 1, 2, 0, 0), sum=6549630855895), Row(time=datetime.datetime(2013, 1, 3, 0, 0), sum=6549630855895), ...] > > Row() objects has two vars: row.time and row.sum > > > The result that I`m looking for is: > [[datetime.datetime(2013, 1, 1, 0, 0), value], > [datetime.datetime(2013, 1, 12, 0, 0), value], > [datetime.datetime(2013, 1, 24, 0, 0), value] > ] > Where value is the sum() of all row.sum in that interval. > > I`m trying to use itertools.groupby by I could not get it to work yet. > Try grouping by the number of days since datetime.datetime(2013, 1, 1, 0, 0), integer-divided by 12. From alister.nospam.ware at ntlworld.com Thu Feb 12 08:29:34 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Thu, 12 Feb 2015 13:29:34 +0000 (UTC) Subject: Group by interval time References: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> Message-ID: On Thu, 12 Feb 2015 04:34:03 -0800, charles.sartori wrote: > Hello there! > > I`m trying to group by a list of Row() objects in 12days interval and > sum(). values. Here is an example of the list > > [Row(time=datetime.datetime(2013, 1, 1, 0, 0), sum=4676557380615), > Row(time=datetime.datetime(2013, 1, 2, 0, 0), sum=6549630855895), > Row(time=datetime.datetime(2013, 1, 3, 0, 0), sum=6549630855895), ...] > > Row() objects has two vars: row.time and row.sum > > > The result that I`m looking for is: > [[datetime.datetime(2013, 1, 1, 0, 0), value], > [datetime.datetime(2013, 1, 12, 0, 0), value], > [datetime.datetime(2013, 1, 24, 0, 0), value] > ] > Where value is the sum() of all row.sum in that interval. > > I`m trying to use itertools.groupby by I could not get it to work yet. > > Thnak you. what is the code you have tried & what errors are you getting? -- life, n.: A whim of several billion cells to be you for a while. From __peter__ at web.de Thu Feb 12 08:29:59 2015 From: __peter__ at web.de (Peter Otten) Date: Thu, 12 Feb 2015 14:29:59 +0100 Subject: Group by interval time References: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> Message-ID: charles.sartori at gmail.com wrote: > Hello there! > > I`m trying to group by a list of Row() objects in 12days interval and > sum(). values. Here is an example of the list > > [Row(time=datetime.datetime(2013, 1, 1, 0, 0), sum=4676557380615), > [Row(time=datetime.datetime(2013, 1, 2, 0, 0), sum=6549630855895), > [Row(time=datetime.datetime(2013, 1, 3, 0, 0), sum=6549630855895), ...] > > Row() objects has two vars: row.time and row.sum > > > The result that I`m looking for is: > [[datetime.datetime(2013, 1, 1, 0, 0), value], > [datetime.datetime(2013, 1, 12, 0, 0), value], > [datetime.datetime(2013, 1, 24, 0, 0), value] > ] > Where value is the sum() of all row.sum in that interval. > > I`m trying to use itertools.groupby by I could not get it to work yet. If the data is sorted by time then you can use groupby, otherwise consider putting it into a (default)dict. Here is an example for both methods: import datetime import random from itertools import groupby from collections import defaultdict, namedtuple BASEDATE = datetime.datetime(2015, 1, 1) INTERVAL = 12 # time interval in days DAYS = datetime.timedelta(days=INTERVAL) def make_sample_rows(): random.seed(42) Row = namedtuple("Row", "time sum") return [ Row(BASEDATE + datetime.timedelta(days=random.randrange(-20, 80)), random.randrange(300)) for i in range(30)] def get_key(row): offset = (row.time - BASEDATE).days // INTERVAL return BASEDATE + datetime.timedelta(days=offset) def format_time(time): return time.strftime("%Y-%m-%d") if __name__ == "__main__": rows = make_sample_rows() # with groupby() for key, group in groupby(sorted(rows), key=get_key): print("{} - {}".format(format_time(key), format_time(key+DAYS))) print("-" * 23) group = list(group) for row in group: print("{} {:4}".format(format_time(row.time), row.sum)) print("{:>15}".format("----")) print("{:15}".format(sum(row.sum for row in group))) print("") # with defaultdict d = defaultdict(int) for row in rows: d[get_key(row)] += row.sum for time, sum in sorted(d.items()): print("{} {:4}".format(format_time(time), sum)) From marko at pacujo.net Thu Feb 12 09:39:53 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Feb 2015 16:39:53 +0200 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> Message-ID: <87twyrchxi.fsf@elektro.pacujo.net> Fabien : > But this was exactly my point! Today in 2015 it's incredibly easy to > write py2/py3 code for a scientist. The whole SciPy track has done the > transition. Not an issue anymore either, for me at least (python > youngster ;-) I write both Py2 and Py3 code, but I keep the two worlds hermetically separated from each other. Marko From megliomortoche at yahoo.com Thu Feb 12 09:44:15 2015 From: megliomortoche at yahoo.com (GIONA.) Date: Thu, 12 Feb 2015 06:44:15 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <20420536-1d7e-4972-b199-d9843b303a32@googlegroups.com> -- redacted -- From charles.sartori at gmail.com Thu Feb 12 09:54:36 2015 From: charles.sartori at gmail.com (charles.sartori at gmail.com) Date: Thu, 12 Feb 2015 06:54:36 -0800 (PST) Subject: Group by interval time In-Reply-To: References: <6f6f567d-7e25-4a74-96ef-d20cb44a9646@googlegroups.com> Message-ID: <07d750f9-04be-457c-9f29-bd4db700aff7@googlegroups.com> Thank you Peter, I was doing wrong at get_key function... Thnak you so much! From rosuav at gmail.com Thu Feb 12 10:59:47 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 02:59:47 +1100 Subject: Python discussed in Nature In-Reply-To: <87twyrchxi.fsf@elektro.pacujo.net> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> Message-ID: On Fri, Feb 13, 2015 at 1:39 AM, Marko Rauhamaa wrote: > Fabien : > >> But this was exactly my point! Today in 2015 it's incredibly easy to >> write py2/py3 code for a scientist. The whole SciPy track has done the >> transition. Not an issue anymore either, for me at least (python >> youngster ;-) > > I write both Py2 and Py3 code, but I keep the two worlds hermetically > separated from each other. One of my students wrote some code for Python 3, and when I went to test it, I typed 'python scriptname' out of habit (most of my students use Py2)... and it almost perfectly worked. The only part that didn't was a class definition that didn't explicitly subclass object, and then used @property. Just changing that class declaration would have made his code 2/3 compatible - and he didn't put any effort into it at all. He literally made his code *accidentally* (almost-)Py2-compatible. You don't need to be afraid of the gap. ChrisA From ian.g.kelly at gmail.com Thu Feb 12 11:17:23 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 12 Feb 2015 09:17:23 -0700 Subject: Alternative to multi-line lambdas: Assign-anywhere def statements In-Reply-To: References: <54C3EAD1.2010501@stoneleaf.us> <54db5356$0$3915$e4fe514c@dreader34.news.xs4all.nl> Message-ID: On Wed, Feb 11, 2015 at 8:56 PM, Chris Angelico wrote: > On Thu, Feb 12, 2015 at 1:57 PM, Ian Kelly wrote: >> The reason I don't like this replacing def isn't because the name is >> necessarily lost. It's because the lack of the well-defined def >> statement encourages more complex usages like >> >> functions['f'] = x -> x**2 >> >> where such implicit transformations won't work. > > That's actually where this started. I mean, if a function can be > called "", why can't it be called "functions['f']"? If the name isn't an identifier then it will make it harder to look up the function from the name via introspection. Another thing that might cause problems is decoration: say_hello = classmethod(cls -> print("Hello from %s" % cls.__name__)) How would the name say_hello make it all the way onto the function object here? I suppose there could be a syntax like: @classmethod say_hello = cls -> print("Hello from %s" % cls.__name__) But now the assignment statement has to be treated grammatically as a special form of assignment, just like a def statement. From ari.brandeis.king at gmail.com Thu Feb 12 11:37:52 2015 From: ari.brandeis.king at gmail.com (Ari King) Date: Thu, 12 Feb 2015 08:37:52 -0800 (PST) Subject: Async/Concurrent HTTP Requests Message-ID: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> Hi, I'd like to query two (or more) RESTful APIs concurrently. What is the pythonic way of doing so? Is it better to use built in functions or are third-party packages? Thanks. Best, Ari From gvanem at yahoo.no Thu Feb 12 11:45:25 2015 From: gvanem at yahoo.no (Gisle Vanem) Date: Thu, 12 Feb 2015 17:45:25 +0100 Subject: Odd version scheme Message-ID: <54DCD8A5.9050907@yahoo.no> I tried using Interactive Python with a PyQt4 console: "IPython.exe qtconsole" But got a "ImportError: IPython requires PyQT4 >= 4.7, found 4.10.4" Looking at Ipython's check (in site-packages\IPython\external\qt.py): if QtCore.PYQT_VERSION_STR < '4.7': raise ImportError("IPython requires PyQt4 >= 4.7, found %s"%QtCore.PYQT_VERSION_STR) So even if '4.10' < '4.7', my '4.10' is newer. What odd version scheme is this really? I just edited that line to read: if QtCore.PYQT_VERSION_STR < '4.10': Seems to be working fine now. -- --gv From ian.g.kelly at gmail.com Thu Feb 12 11:56:25 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 12 Feb 2015 09:56:25 -0700 Subject: Odd version scheme In-Reply-To: <54DCD8A5.9050907@yahoo.no> References: <54DCD8A5.9050907@yahoo.no> Message-ID: On Thu, Feb 12, 2015 at 9:45 AM, Gisle Vanem wrote: > I tried using Interactive Python with a PyQt4 console: > "IPython.exe qtconsole" > > But got a > "ImportError: IPython requires PyQT4 >= 4.7, found 4.10.4" > > Looking at Ipython's check (in site-packages\IPython\external\qt.py): > if QtCore.PYQT_VERSION_STR < '4.7': > raise ImportError("IPython requires PyQt4 >= 4.7, found > %s"%QtCore.PYQT_VERSION_STR) > > So even if '4.10' < '4.7', my '4.10' is newer. What odd version scheme is > this really? I just edited that line to read: > if QtCore.PYQT_VERSION_STR < '4.10': > > Seems to be working fine now. There's nothing wrong with the version scheme; 4.10 is a greater version than 4.7. The problem appears to be IPython using a simple lexicographical string comparison to compare the version numbers and failing when it gets to a number that isn't a single digit. From marko at pacujo.net Thu Feb 12 11:56:46 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Feb 2015 18:56:46 +0200 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> Message-ID: <87oaozcbld.fsf@elektro.pacujo.net> Chris Angelico : > On Fri, Feb 13, 2015 at 1:39 AM, Marko Rauhamaa wrote: >> I write both Py2 and Py3 code, but I keep the two worlds hermetically >> separated from each other. > > [...] > > You don't need to be afraid of the gap. No problem. When I write Py3, I write Py3. When I write Py2, I write Py2. When I write bash, I write bash. When I write C, I write C. Who's afraid of the Big Bad Wolf: ======================================================================== $ python3 Python 3.4.1 (default, Nov 3 2014, 14:38:10) [GCC 4.9.1 20140930 (Red Hat 4.9.1-11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path = [ '/usr/lib64/python2.7' ] + sys.path >>> import re Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python2.7/re.py", line 240 raise TypeError, "first argument must be string or compiled pattern" ^ SyntaxError: invalid syntax >>> import xmllib Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python2.7/xmllib.py", line 813 print 'xml: encoding =',encoding,'standalone =',standalone ^ SyntaxError: invalid syntax ======================================================================== Marko From rosuav at gmail.com Thu Feb 12 11:58:36 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 03:58:36 +1100 Subject: Odd version scheme In-Reply-To: <54DCD8A5.9050907@yahoo.no> References: <54DCD8A5.9050907@yahoo.no> Message-ID: On Fri, Feb 13, 2015 at 3:45 AM, Gisle Vanem wrote: > Looking at Ipython's check (in site-packages\IPython\external\qt.py): > if QtCore.PYQT_VERSION_STR < '4.7': > raise ImportError("IPython requires PyQt4 >= 4.7, found > %s"%QtCore.PYQT_VERSION_STR) > > So even if '4.10' < '4.7', my '4.10' is newer. What odd version scheme is > this really? I just edited that line to read: > if QtCore.PYQT_VERSION_STR < '4.10': Looks to me like a fault in comparison format. Normally, you'd expect that 4.10 follows 4.9 - that is, that the version number is a tuple of integers - but this is attempting a naive string comparison on it. Check for an ipython bug tracker and report the matter; that ought to be fixed. Bear in mind that your change now means that you're accepting too much - for instance. '4.6' is not less than '4.10', even though it's supposed to be rejecting that. Presumably this won't be a problem on your own system, but for upstream, a more reliable fix will be needed. ChrisA From python.list at tim.thechases.com Thu Feb 12 12:05:06 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 12 Feb 2015 11:05:06 -0600 Subject: Odd version scheme In-Reply-To: <54DCD8A5.9050907@yahoo.no> References: <54DCD8A5.9050907@yahoo.no> Message-ID: <20150212110506.39410d54@bigbox.christie.dr> On 2015-02-12 17:45, Gisle Vanem wrote: > I tried using Interactive Python with a PyQt4 console: > "IPython.exe qtconsole" > > But got a > "ImportError: IPython requires PyQT4 >= 4.7, found 4.10.4" > > Looking at Ipython's check (in > site-packages\IPython\external\qt.py): if QtCore.PYQT_VERSION_STR < > '4.7': raise ImportError("IPython requires PyQt4 >= 4.7, found > %s"%QtCore.PYQT_VERSION_STR) > > So even if '4.10' < '4.7', my '4.10' is newer. What odd version > scheme is this really? I just edited that line to read: > if QtCore.PYQT_VERSION_STR < '4.10':' This looks like it's comparing them as strings. It would be wiser if the program compared them as version-numbers: from distutils.version import LooseVersion, StrictVersion qt_ver = QtCore.PYQT_VERSION_STR target_ver = "4.10" for t, desc in [ (str, "String"), (LooseVersion, "Loose"), (StrictVersion, "Strict"), ]: a = t(qt_ver) b = t(target_ver) print("Using %s comparison, '%r < %r' is %s" % ( desc, qt_ver, target_ver, a < b) So the test should actually be something like if LooseVersion(QtCore.PYQT_VERSION_STR) < LooseVersion("4.10"): balk() -tkc From rosuav at gmail.com Thu Feb 12 12:08:26 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 04:08:26 +1100 Subject: Python discussed in Nature In-Reply-To: <87oaozcbld.fsf@elektro.pacujo.net> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> <87oaozcbld.fsf@elektro.pacujo.net> Message-ID: On Fri, Feb 13, 2015 at 3:56 AM, Marko Rauhamaa wrote: > $ python3 > Python 3.4.1 (default, Nov 3 2014, 14:38:10) > [GCC 4.9.1 20140930 (Red Hat 4.9.1-11)] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> sys.path = [ '/usr/lib64/python2.7' ] + sys.path Interesting for intellectual curiosity, but not particularly significant. All you prove is that the 2.7 stdlib is designed for compatibility with older 2.x Pythons (or, looking the other way, nobody wants pointless code churn in working code). The best way to write 2/3 compatible code is to write 3.x code first. Then just adhere to a few simple rules (eg always subclass object), and the bulk of your code will work just fine. You might need to toss in some explicit __future__ directives (print_function if you don't stick to "exactly one string argument to print", and division if you don't want to have to worry about the differences), and if you do this a lot, you might want to pick up the 'six' library or equivalent, but other than that, there's really not much to deal with. ChrisA From skip.montanaro at gmail.com Thu Feb 12 12:19:19 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 12 Feb 2015 11:19:19 -0600 Subject: Odd version scheme In-Reply-To: <20150212110506.39410d54@bigbox.christie.dr> References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: I believe this sort of lexicographical comparison wart is one of the reasons the Python-dev gang decided that there would be no micro versions > 9. There are too many similar assumptions about version numbers out in the real world. Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 12 12:26:08 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 04:26:08 +1100 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: On Fri, Feb 13, 2015 at 4:19 AM, Skip Montanaro wrote: > I believe this sort of lexicographical comparison wart is one of the reasons > the Python-dev gang decided that there would be no micro versions > 9. There > are too many similar assumptions about version numbers out in the real > world. I don't think the policy is quite that strong; 2.7.10 has already been scheduled, and there's sufficient resistance to a Python 4.0 that there's likely to be a 3.10; although some people are saying that it might be worth following 3.9 with 4.0, drop only stuff that's been deprecated for a long time (on par with the disallowing of string exceptions during 2.x), and basically dodge the whole issue. But yes, there's a general feeling that two-digit version components cause trouble, and it's usually easy enough to avoid them. ChrisA From ian.g.kelly at gmail.com Thu Feb 12 12:35:53 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 12 Feb 2015 10:35:53 -0700 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: On Thu, Feb 12, 2015 at 10:19 AM, Skip Montanaro wrote: > I believe this sort of lexicographical comparison wart is one of the reasons > the Python-dev gang decided that there would be no micro versions > 9. There > are too many similar assumptions about version numbers out in the real > world. It still becomes an issue when we get to Python 10. From gvanem at yahoo.no Thu Feb 12 12:37:29 2015 From: gvanem at yahoo.no (Gisle Vanem) Date: Thu, 12 Feb 2015 18:37:29 +0100 Subject: Odd version scheme In-Reply-To: <20150212110506.39410d54@bigbox.christie.dr> References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: <54DCE4D9.2030100@yahoo.no> Tim Chase wrote:> So the test should actually be something like > if LooseVersion(QtCore.PYQT_VERSION_STR) < LooseVersion("4.10"): > balk() That's exactly what they do now in IPython/utils/version.py with the comment: Utilities for version comparison It is a bit ridiculous that we need these. Not sure why this is "ridiculous". I had an "old" version of IPython v0.13 installed! I've upgraded to v2.10 and all is good. -- --gv From python.list at tim.thechases.com Thu Feb 12 12:46:31 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 12 Feb 2015 11:46:31 -0600 Subject: Odd version scheme In-Reply-To: <54DCE4D9.2030100@yahoo.no> References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCE4D9.2030100@yahoo.no> Message-ID: <20150212114631.7b132fed@bigbox.christie.dr> On 2015-02-12 18:37, Gisle Vanem wrote: > Tim Chase wrote:> So the test should actually be something like > > > if LooseVersion(QtCore.PYQT_VERSION_STR) < > > LooseVersion("4.10"): balk() > > That's exactly what they do now in IPython/utils/version.py with > the comment: > Utilities for version comparison > It is a bit ridiculous that we need these. > > Not sure why this is "ridiculous". Well, if you're storing them as strings, then duh, you need the helper utilities. If you store them as comparable tuples, then it's not a problem. :-/ # library.VERSION = (3, 14, 15) if library.VERSION < (2, 3, 18): complain() It does become harder to mark things as alpha/beta/pre/rc, but something Enum-like might suffice for that in a fourth field of the tuple. -tkc From zachary.ware+pylist at gmail.com Thu Feb 12 12:59:41 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Thu, 12 Feb 2015 11:59:41 -0600 Subject: Async/Concurrent HTTP Requests In-Reply-To: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> References: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> Message-ID: On Thu, Feb 12, 2015 at 10:37 AM, Ari King wrote: > Hi, > > I'd like to query two (or more) RESTful APIs concurrently. What is the pythonic way of doing so? Is it better to use built in functions or are third-party packages? Thanks. Have a look at asyncio (new in Python 3.4, available for 3.3 as the 'tulip' project) and possibly the aiohttp project, available on PyPI. I'm using both for a current project, and they work very well. -- Zach From no.email at nospam.invalid Thu Feb 12 13:15:21 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 12 Feb 2015 10:15:21 -0800 Subject: Async/Concurrent HTTP Requests References: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> Message-ID: <87y4o3f13a.fsf@jester.gateway.pace.com> Ari King writes: > I'd like to query two (or more) RESTful APIs concurrently. What is the > pythonic way of doing so? Is it better to use built in functions or > are third-party packages? Thanks. The two basic approaches are event-based asynchronous i/o (there are various packages for that) and threads. There are holy wars over which is better. Event-driven i/o in Python 2.x was generally done with callback-based packages like Twisted Matrix (www.twistedmatrix.com). In Python 3 there are some nicer mechanisms (coroutines) so the new asyncio package may be easier to use than Twisted. I haven't tried it yet. From john_ladasky at sbcglobal.net Thu Feb 12 13:29:33 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 12 Feb 2015 10:29:33 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> Message-ID: <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> On Thursday, February 12, 2015 at 3:08:10 AM UTC-8, Fabien wrote: > ... what a coincidence then that a huge majority of scientists > (including me) dont care AT ALL about unicode. But since scientists are > not paid to rewrite old code, the scientific world is still stuck to > python 2. I'm a scientist. I'm a happy Python 3 user who migrated from Python 2 about two years ago. And I use Unicode in my Python. In implementing some mathematical models which have variables like delta, gamma, and theta, I decided that I didn't like the line lengths I was getting with such variable names. I'm using ?, ?, and ? instead. It works fine, at least on my Ubuntu Linux system (and what scientist doesn't use Linux?). I also have special mathematical symbols, superscripted numbers, etc. in my program comments. It's easier to read 2x? + 3x? than 2*x**3 + 3*x**2. I am teaching someone Python who is having a few problems with Unicode on his Windows 7 machine. It would appear that Windows shipped with a less-than-complete Unicode font for its command shell. But that's not Python's fault. From rosuav at gmail.com Thu Feb 12 13:37:43 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 05:37:43 +1100 Subject: Python discussed in Nature In-Reply-To: <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> Message-ID: On Fri, Feb 13, 2015 at 5:29 AM, John Ladasky wrote: > It works fine, at least on my Ubuntu Linux system (and what scientist doesn't use Linux?). I also have special mathematical symbols, superscripted numbers, etc. in my program comments. It's easier to read 2x? + 3x? than 2*x**3 + 3*x**2. > > I am teaching someone Python who is having a few problems with Unicode on his Windows 7 machine. It would appear that Windows shipped with a less-than-complete Unicode font for its command shell. But that's not Python's fault. > Yes, Windows's default terminal/console does have issues. If all your text is staying within the BMP, you may be able to run it within IDLE to get somewhat better results; or PowerShell may help. But as you say, that's not Python's fault. Fortunately, it's not difficult to write a GUI program that manipulates Unicode text, or something that works entirely with files and leaves the display up to something else (maybe a good text editor, or a web browser). All your internals are working perfectly, it's just the human interface that's a bit harder. And only on flawed/broken platforms. ChrisA From ethan at stoneleaf.us Thu Feb 12 13:48:34 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 12 Feb 2015 10:48:34 -0800 Subject: Python discussed in Nature In-Reply-To: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54DCF582.3060804@stoneleaf.us> On 02/12/2015 12:46 AM, Steven D'Aprano wrote: > > "Nature", one of the world's premier science journals, has published an > excellent article about programming in Python: > > http://www.nature.com/news/programming-pick-up-python-1.16833 That is a very nice article, thanks for sharing! -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From marko at pacujo.net Thu Feb 12 13:55:07 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Feb 2015 20:55:07 +0200 Subject: Async/Concurrent HTTP Requests References: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> <87y4o3f13a.fsf@jester.gateway.pace.com> Message-ID: <87k2znufhw.fsf@elektro.pacujo.net> Paul Rubin : > Event-driven i/o in Python 2.x was generally done with callback-based > packages like Twisted Matrix (www.twistedmatrix.com). In Python 3 > there are some nicer mechanisms (coroutines) so the new asyncio > package may be easier to use than Twisted. I haven't tried it yet. I have successfully done event-driven I/O using select.epoll() and socket.socket(). Marko From no.email at nospam.invalid Thu Feb 12 13:57:18 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 12 Feb 2015 10:57:18 -0800 Subject: Async/Concurrent HTTP Requests References: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> <87y4o3f13a.fsf@jester.gateway.pace.com> <87k2znufhw.fsf@elektro.pacujo.net> Message-ID: <87twyrez5d.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > I have successfully done event-driven I/O using select.epoll() and > socket.socket(). Sure, but then you end up writing a lot of low-level machinery that packages like twisted take care of for you. From python at mrabarnett.plus.com Thu Feb 12 13:58:30 2015 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 12 Feb 2015 18:58:30 +0000 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: <54DCF7D6.9090606@mrabarnett.plus.com> On 2015-02-12 17:35, Ian Kelly wrote: > On Thu, Feb 12, 2015 at 10:19 AM, Skip Montanaro > wrote: >> I believe this sort of lexicographical comparison wart is one of the reasons >> the Python-dev gang decided that there would be no micro versions > 9. There >> are too many similar assumptions about version numbers out in the real >> world. > > It still becomes an issue when we get to Python 10. > Just call it Python X! :-) From ian.g.kelly at gmail.com Thu Feb 12 14:16:25 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 12 Feb 2015 12:16:25 -0700 Subject: Odd version scheme In-Reply-To: <54DCF7D6.9090606@mrabarnett.plus.com> References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCF7D6.9090606@mrabarnett.plus.com> Message-ID: On Thu, Feb 12, 2015 at 11:58 AM, MRAB wrote: > On 2015-02-12 17:35, Ian Kelly wrote: >> >> On Thu, Feb 12, 2015 at 10:19 AM, Skip Montanaro >> wrote: >>> >>> I believe this sort of lexicographical comparison wart is one of the >>> reasons >>> the Python-dev gang decided that there would be no micro versions > 9. >>> There >>> are too many similar assumptions about version numbers out in the real >>> world. >> >> >> It still becomes an issue when we get to Python 10. >> > Just call it Python X! :-) Things break down again when we get to Python XIX. >>> 'XVIII' < 'XIX' False From breamoreboy at yahoo.co.uk Thu Feb 12 14:24:31 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 12 Feb 2015 19:24:31 +0000 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCF7D6.9090606@mrabarnett.plus.com> Message-ID: On 12/02/2015 19:16, Ian Kelly wrote: > On Thu, Feb 12, 2015 at 11:58 AM, MRAB wrote: >> On 2015-02-12 17:35, Ian Kelly wrote: >>> >>> On Thu, Feb 12, 2015 at 10:19 AM, Skip Montanaro >>> wrote: >>>> >>>> I believe this sort of lexicographical comparison wart is one of the >>>> reasons >>>> the Python-dev gang decided that there would be no micro versions > 9. >>>> There >>>> are too many similar assumptions about version numbers out in the real >>>> world. >>> >>> >>> It still becomes an issue when we get to Python 10. >>> >> Just call it Python X! :-) > > Things break down again when we get to Python XIX. > >>>> 'XVIII' < 'XIX' > False > I believe that this could be solved by borrowing from Mark Pilgrim's excellent "Dive Into Python" which uses (or used?) these hex (?) numbers as the basis for a look at unit testing. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python.list at tim.thechases.com Thu Feb 12 14:28:01 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 12 Feb 2015 13:28:01 -0600 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCF7D6.9090606@mrabarnett.plus.com> Message-ID: <20150212132801.3f02e687@bigbox.christie.dr> On 2015-02-12 12:16, Ian Kelly wrote: > >> It still becomes an issue when we get to Python 10. > >> > > Just call it Python X! :-) > > Things break down again when we get to Python XIX. > > >>> 'XVIII' < 'XIX' > False You know what this sub-thread gives me? The icks. https://www.youtube.com/watch?v=6DzfPcSysAg -tkc From marko at pacujo.net Thu Feb 12 14:37:09 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Feb 2015 21:37:09 +0200 Subject: Async/Concurrent HTTP Requests References: <7a6ca7b9-ad3a-4361-88fc-580e41452874@googlegroups.com> <87y4o3f13a.fsf@jester.gateway.pace.com> <87k2znufhw.fsf@elektro.pacujo.net> <87twyrez5d.fsf@jester.gateway.pace.com> Message-ID: <87fvaavs4a.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> I have successfully done event-driven I/O using select.epoll() and >> socket.socket(). > > Sure, but then you end up writing a lot of low-level machinery that > packages like twisted take care of for you. Certainly. It would be nice if the stdlib protocol facilities were event-driven and divorced from the low-level I/O. Asyncio does that, of course, but the programming model feels a bit weird. Twisted documentation seems a bit vague on details. For example, what should one make of this: def write(data): Write some data to the physical connection, in sequence, in a non-blocking fashion. If possible, make sure that it is all written. No data will ever be lost, although (obviously) the connection may be closed before it all gets through. So I'm left wondering if the call will block and if not, how is flow control and buffering managed. The API documentation leads me to a maze of twisted passages, all alike. From what I could gather, the write() method is blocking and hence not suitable for serious work. By contrast, the semantics of Python's socket.send() is crisply defined and a pleasure to work with. Marko From taleinat at gmail.com Thu Feb 12 14:51:17 2015 From: taleinat at gmail.com (Tal Einat) Date: Thu, 12 Feb 2015 21:51:17 +0200 Subject: fuzzysearch: find not exactly what you're looking for! Message-ID: Hi everyone, I'd like to introduce a Python library I've been working on for a while: fuzzysearch. I would love to get as much feedback as possible: comments, suggestions, bugs and more are all very welcome! fuzzysearch is useful for searching when you'd like to find nearly-exact matches. What should be considered a "nearly matching" sub-string is defined by a maximum allowed Levenshtein distance[1]. This can be further refined by indicating the maximum allowed number of substitutions, insertions and/or deletions, each separately. Here is a basic example: >>> from fuzzysearch import find_near_matches >>> find_near_matches('PATTERN', 'aaaPATERNaaa', max_l_dist=1) [Match(start=3, end=9, dist=1)] The library supports Python 2.6+ and 3.2+ with a single code base. It is extensively tested with 97% code coverage. There are many optimizations under the hood, including custom algorithms and C extensions implemented in C and Cython. Install as usual: $ pip install fuzzysearch The repo is on github: https://github.com/taleinat/fuzzysearch Let me know what you think! - Tal Einat .. [1]: http://en.wikipedia.org/wiki/Levenshtein_distance From hniksic at gmail.com Thu Feb 12 15:23:10 2015 From: hniksic at gmail.com (=?UTF-8?B?SHJ2b2plIE5pa8WhacSH?=) Date: Thu, 12 Feb 2015 21:23:10 +0100 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: > >>>>> from decimal import Decimal as D > >>>>> x = D(1)/D(999) > >>>>> '{:.15g}'.format(x) > >> > >> '0.00100100100100100' [...] > > I'd say it's a bug. P is 15, you've got 17 digits after the decimal place > > and two of those are insignificant trailing zeros. > > Actually it's the float version that doesn't match the documentation. > In the decimal version, sure there are 17 digits after the decimal > place there, but the first two -- which are leading zeroes -- would > not normally be considered significant. {:.15g} is supposed to give 15 digits of precision, but with trailing zeros removed. For example, '{:.15g}'.format(Decimal('0.5')) should yield '0.5', not '0.500000000000000' -- and, it indeed does. It is only for some numbers that trailing zeros are not removed, which looks like a bug. The behavior of floats matches both the documentation and other languages using the 'g' decimal format, such as C. > The float version OTOH is only giving you 13 significant digits when > 15 were requested. It is giving 15 significant digits if you count the trailing zeros that have been removed. If those two digits had not been zeros, they would have been included. This is again analogous to '{:.15g}'.format(0.5) returning '0.5'. From emile at fenx.com Thu Feb 12 15:54:06 2015 From: emile at fenx.com (Emile van Sebille) Date: Thu, 12 Feb 2015 12:54:06 -0800 Subject: fuzzysearch: find not exactly what you're looking for! In-Reply-To: References: Message-ID: On 2/12/2015 11:51 AM, Tal Einat wrote: > Hi everyone, > > I'd like to introduce a Python library I've been working on for a > while: fuzzysearch. I would love to get as much feedback as possible: > comments, suggestions, bugs and more are all very welcome! I adapt difflib's SequenceMatcher for my fuzzy search needs now -- can you provide some perspective on how fuzzysearch compares? Thanks, Emile From emile at fenx.com Thu Feb 12 16:08:09 2015 From: emile at fenx.com (Emile van Sebille) Date: Thu, 12 Feb 2015 13:08:09 -0800 Subject: Odd version scheme In-Reply-To: References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCF7D6.9090606@mrabarnett.plus.com> Message-ID: On 2/12/2015 11:16 AM, Ian Kelly wrote: > Things break down again when we get to Python XIX. >>>> 'XVIII' < 'XIX' > False Looks to me like you better check if your PEP313 patch is installed properly. :) Emile From tjreedy at udel.edu Thu Feb 12 16:12:31 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Feb 2015 16:12:31 -0500 Subject: Bad text appearance in IDLE In-Reply-To: <54dc9bee$0$3046$426a34cc@news.free.fr> References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: On 2/12/2015 7:26 AM, ast wrote: > Hello > > Here is how text appears in IDLE window > http://www.cjoint.com/data/0BmnEIcxVAx.htm Do you get anything similar when running the console interpreter? > Yesterday evening I had not this trouble. It appears > this morning. I restarted my computer with no effect. > > A windows Vista update has been done this morning, with about 10 fixes. > I suspect something gone wrong with this update > > Has somebody an explanation about that ? Assuming not, the difference between them is that Idle writes to the screen via a tkinter (tk) text widget. You might try something like this to see if tkinter is generally not working, even when run without Idle. import tkinter as tk root = tk.Tk() text = tk.Text() text.pack() text.insert('1.0', 'this is a test eeeccee') -- Terry Jan Reedy From taleinat at gmail.com Thu Feb 12 18:08:54 2015 From: taleinat at gmail.com (Tal Einat) Date: Thu, 12 Feb 2015 23:08:54 +0000 (UTC) Subject: fuzzysearch: find not exactly what you're looking for! References: Message-ID: Emile van Sebille writes: > I adapt difflib's SequenceMatcher for my fuzzy search needs now -- can > you provide some perspective on how fuzzysearch compares? Hi Emile, fuzzysearch is made for this purpose, is straightforward to use, is better documented and is much faster. What do you mean by "adapt difflib's SequenceMatcher"? If you mean that you alter its code, then certainly there is value in a library which provides such functionality without requiring any adaptation? - Tal Einat From taleinat at gmail.com Thu Feb 12 18:19:44 2015 From: taleinat at gmail.com (Tal Einat) Date: Fri, 13 Feb 2015 01:19:44 +0200 Subject: fuzzysearch: find not exactly what you're looking for! Message-ID: Emile van Sebille writes: > I adapt difflib's SequenceMatcher for my fuzzy search needs now -- can > you provide some perspective on how fuzzysearch compares? Hi Emile, fuzzysearch is made for this purpose, is straightforward to use, is better documented and is much faster. What do you mean by "adapt difflib's SequenceMatcher"? If you mean that you alter its code, then certainly there is value in a library which provides such functionality without requiring any adaptation? - Tal Einat From ian.g.kelly at gmail.com Thu Feb 12 18:46:07 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 12 Feb 2015 16:46:07 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: >> >>>>> from decimal import Decimal as D >> >>>>> x = D(1)/D(999) >> >>>>> '{:.15g}'.format(x) >> >> >> >> '0.00100100100100100' > [...] >> > I'd say it's a bug. P is 15, you've got 17 digits after the decimal place >> > and two of those are insignificant trailing zeros. >> >> Actually it's the float version that doesn't match the documentation. >> In the decimal version, sure there are 17 digits after the decimal >> place there, but the first two -- which are leading zeroes -- would >> not normally be considered significant. > > {:.15g} is supposed to give 15 digits of precision, but with trailing > zeros removed. The doc says with "insignificant" trailing zeros removed, not all trailing zeros. > For example, '{:.15g}'.format(Decimal('0.5')) should > yield '0.5', not '0.500000000000000' -- and, it indeed does. It is > only for some numbers that trailing zeros are not removed, which looks > like a bug. The behavior of floats matches both the documentation and > other languages using the 'g' decimal format, such as C. Ah, I see now what's going on here. With floats, there is really no notion of significant digits. The values 0.5 and 0.50000 are completely equivalent. With decimals, that's not exactly true; if you give the decimal a trailing zero then you are telling it that the zero is significant. >>> Decimal('0.5') Decimal('0.5') >>> Decimal('0.50000') Decimal('0.50000') >>> Decimal('0.5').as_tuple() DecimalTuple(sign=0, digits=(5,), exponent=-1) >>> Decimal('0.50000').as_tuple() DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5) These are distinct; the decimal knows how many significant digits you passed it. As a result, these are also distinct: >>> '{:.4g}'.format(Decimal('0.5')) '0.5' >>> '{:.4g}'.format(Decimal('0.50000')) '0.5000' Now what happens in your original example of 1/999? The default decimal context uses 28 digits of precision, so the result of that calculation will have 28 significant digits in it. >>> decimal.getcontext().prec 28 >>> Decimal(1) / Decimal(999) Decimal('0.001001001001001001001001001001') When you specify the a precision of 15 in your format string, you're telling it to take the first 15 of those. It doesn't care that the last couple of those are zeros, because as far as it's concerned, those digits are significant. From polleysarah7 at gmail.com Thu Feb 12 19:07:00 2015 From: polleysarah7 at gmail.com (polleysarah7 at gmail.com) Date: Thu, 12 Feb 2015 16:07:00 -0800 (PST) Subject: Download multiple xls files using Python Message-ID: <5e72ce17-3bbd-4b98-a81a-9a0e21120445@googlegroups.com> I was wondering if somebody here could help me out creating a script? I have never done something like this before so I have no idea what I'm doing. But I have been reading about it for a couple days now and I'm still not understanding it so I appreciating all help I can get. I'm even willing to pay for your service! Here is an example of my problem. I have for the moment a CSV file named "Stars" saved on my windows desktop containing around 50.000 different links that directly starts downloading a xls file when pressed. Each row contains one of these links. I would want with your help create some kind of script for this that will make some kind of loop thru each row and visit this different links so it can download these 50.000 different files. Thank you all for taking time to read this / Sarah From polleysarah7 at gmail.com Thu Feb 12 19:07:43 2015 From: polleysarah7 at gmail.com (polleysarah7 at gmail.com) Date: Thu, 12 Feb 2015 16:07:43 -0800 (PST) Subject: Download multiple xls files using Python Message-ID: <4af26ab8-a9a2-4045-8c62-712f593f98b0@googlegroups.com> I was wondering if somebody here could help me out creating a script? I have never done something like this before so I have no idea what I'm doing. But I have been reading about it for a couple days now and I'm still not understanding it so I appreciating all help I can get. I'm even willing to pay for your service! Here is an example of my problem. I have for the moment a CSV file named "Stars" saved on my windows desktop containing around 50.000 different links that directly starts downloading a xls file when pressed. Each row contains one of these links. I would want with your help create some kind of script for this that will make some kind of loop thru each row and visit this different links so it can download these 50.000 different files. Thank you all for taking time to read this / Sarah From rosuav at gmail.com Thu Feb 12 19:19:20 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Feb 2015 11:19:20 +1100 Subject: Download multiple xls files using Python In-Reply-To: <5e72ce17-3bbd-4b98-a81a-9a0e21120445@googlegroups.com> References: <5e72ce17-3bbd-4b98-a81a-9a0e21120445@googlegroups.com> Message-ID: On Fri, Feb 13, 2015 at 11:07 AM, wrote: > Here is an example of my problem. I have for the moment a CSV file named "Stars" saved on my windows desktop containing around 50.000 different links that directly starts downloading a xls file when pressed. Each row contains one of these links. I would want with your help create some kind of script for this that will make some kind of loop thru each row and visit this different links so it can download these 50.000 different files. > Let's break that down into pieces. 1) You have a CSV file, from which you'd like to extract specific data (URLs). 2) You have a series of URLs, which you would like to download. Python can do both of these jobs! The first is best handled by the csv module, and the second by the requests module (which you'd have to get off PyPI). https://docs.python.org/3/library/csv.html http://docs.python-requests.org/en/latest/ In short: Loop over the lines in the file, as returned by the CSV reader, and for each line, fire off a request to fetch its XLS file, and save it to disk. Given that you're working with 50K distinct URLs, you'll want to have some kind of check to see if you already have the file, so you can restart the downloader script. Have fun! ChrisA From python.list at tim.thechases.com Thu Feb 12 20:14:46 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 12 Feb 2015 19:14:46 -0600 Subject: Download multiple xls files using Python In-Reply-To: References: <5e72ce17-3bbd-4b98-a81a-9a0e21120445@googlegroups.com> Message-ID: <20150212191446.4d60a964@bigbox.christie.dr> On 2015-02-13 11:19, Chris Angelico wrote: > On Fri, Feb 13, 2015 at 11:07 AM, wrote: > > Here is an example of my problem. I have for the moment a CSV > > file named "Stars" saved on my windows desktop containing around > > 50.000 different links that directly starts downloading a xls > > file when pressed. Each row contains one of these links. I would > > want with your help create some kind of script for this that will > > make some kind of loop thru each row and visit this different > > links so it can download these 50.000 different files. > > In short: Loop over the lines in the file, as returned by the CSV > reader, and for each line, fire off a request to fetch its XLS file, Alternatively, unless you're wed to Python, downloading tools like "wget" are your friend and make quick work of this. Just extract the column of URLs into a file (which can be done in Excel or with any number of utilities such as "cut") and then use "wget" to pull them all down: cut -d, -f3 < data.csv > urls.txt wget -i urls.txt (the above assumes that column #3 contains the URLs) -tkc From ben+python at benfinney.id.au Thu Feb 12 20:20:14 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Feb 2015 12:20:14 +1100 Subject: Parsing and comparing version strings (was: Odd version scheme) References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCE4D9.2030100@yahoo.no> Message-ID: <85k2zmd2up.fsf_-_@benfinney.id.au> Gisle Vanem writes: > That's exactly what they do now in IPython/utils/version.py with > the comment: > Utilities for version comparison > It is a bit ridiculous that we need these. > > Not sure why this is "ridiculous". Right, versions are effectively a special type [0], specifically *because* they intentionally don't compare as scalar numbers or strings. It's not ?ridiculous? to need custom comparisons when that's the case. Python even comes with support for version parsing and comparison in the standard library [1]. So if anything's ?ridiculous?, it's the act of re-implementing that and getting it wrong. (Or maybe that such an important part of the standard library is largely undocumented.) [0] [1] Unfortunately undocumented, like much of the Distutils API. Try . -- \ ?Programs must be written for people to read, and only | `\ incidentally for machines to execute.? ?Abelson & Sussman, | _o__) _Structure and Interpretation of Computer Programs_ | Ben Finney From python.list at tim.thechases.com Thu Feb 12 20:34:50 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 12 Feb 2015 19:34:50 -0600 Subject: Parsing and comparing version strings (was: Odd version scheme) In-Reply-To: <85k2zmd2up.fsf_-_@benfinney.id.au> References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> <54DCE4D9.2030100@yahoo.no> <85k2zmd2up.fsf_-_@benfinney.id.au> Message-ID: <20150212193450.470d4a47@bigbox.christie.dr> On 2015-02-13 12:20, Ben Finney wrote: > > Not sure why this is "ridiculous". > > Right, versions are effectively a special type [0], specifically > *because* they intentionally don't compare as scalar numbers or > strings. It's not ?ridiculous? to need custom comparisons when > that's the case. > > Python even comes with support for version parsing and comparison > in the standard library [1]. So if anything's ?ridiculous?, it's > the act of re-implementing that and getting it wrong. > > (Or maybe that such an important part of the standard library is > largely undocumented.) I was surprised (pleasantly) to learn of that corner of distutils, but can never remember where it is in the stdlib or what it's called. So it's pretty standard for my process to be something like - Hmm, I need to compare version-strings - search the web for "python compare version number" - note the top-ranked Stack Overflow answer - spot that it uses distutils.version.{LooseVersion,StrictVersion} - use that in my code, optionally searching to get full docs with "site:python.org LooseVersion StrictVersion", only to be surprised when something like https://docs.python.org/2/library/{module} isn't anywhere in the top umpteen hits. "largely undocumented" is an understatement :-) -tkc From steve+comp.lang.python at pearwood.info Thu Feb 12 22:07:46 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 13 Feb 2015 14:07:46 +1100 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> <87oaozcbld.fsf@elektro.pacujo.net> Message-ID: <54dd6a83$0$12998$c3e8da3$5496439d@news.astraweb.com> Marko Rauhamaa wrote: > Chris Angelico : > >> On Fri, Feb 13, 2015 at 1:39 AM, Marko Rauhamaa wrote: >>> I write both Py2 and Py3 code, but I keep the two worlds hermetically >>> separated from each other. >> >> [...] >> >> You don't need to be afraid of the gap. > > No problem. When I write Py3, I write Py3. When I write Py2, I write > Py2. When I write bash, I write bash. When I write C, I write C. Do you get confused by the difference between talking to Americans and talking to Britons? The differences between American and British English is a better analogy for the differences between Python 2 and 3 than the differences between C and bash. Especially if you target 2.7 and 3.3+, it is almost trivially easy to write multi-dialect Python 2 and 3 code in the same code base. The trickiest part is not a language change at all, but remembering that the names of some standard library modules have changed. Nobody here is suggesting that there are no differences between Python 2 and 3, but suggesting that those differences are of the same order of magnitude as those between bash and C is ridiculous. The common subset of the Python language is far greater than the differences between the versions. -- Steven From steve+comp.lang.python at pearwood.info Thu Feb 12 22:10:42 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 13 Feb 2015 14:10:42 +1100 Subject: Odd version scheme References: <54DCD8A5.9050907@yahoo.no> <20150212110506.39410d54@bigbox.christie.dr> Message-ID: <54dd6b32$0$12998$c3e8da3$5496439d@news.astraweb.com> Skip Montanaro wrote: > I believe this sort of lexicographical comparison wart is one of the > reasons the Python-dev gang decided that there would be no micro versions > > 9. There are too many similar assumptions about version numbers out in > the real world. Which is why there will be no Windows 9. Too many bad programmers that test for Windows 95 or 98 by checking the version string and doing a prefix test for "Windows 9". -- Steven From steve+comp.lang.python at pearwood.info Thu Feb 12 22:11:50 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 13 Feb 2015 14:11:50 +1100 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> Message-ID: <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> John Ladasky wrote: > And I use Unicode in my Python. In implementing some mathematical models > which have variables like delta, gamma, and theta, I decided that I didn't > like the line lengths I was getting with such variable names. I'm using > ?, ?, and ? instead. It works fine, at least on my Ubuntu Linux system > (and what scientist doesn't use Linux?). I also have special mathematical > symbols, superscripted numbers, etc. in my program comments. It's easier > to read 2x? + 3x? than 2*x**3 + 3*x**2. Oooh! What is your font of choice for this? -- Steven From rustompmody at gmail.com Thu Feb 12 23:07:26 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 12 Feb 2015 20:07:26 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> Message-ID: <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> On Thursday, February 12, 2015 at 11:59:55 PM UTC+5:30, John Ladasky wrote: > On Thursday, February 12, 2015 at 3:08:10 AM UTC-8, Fabien wrote: > > > ... what a coincidence then that a huge majority of scientists > > (including me) dont care AT ALL about unicode. But since scientists are > > not paid to rewrite old code, the scientific world is still stuck to > > python 2. > > I'm a scientist. I'm a happy Python 3 user who migrated from Python 2 about two years ago. > > And I use Unicode in my Python. In implementing some mathematical models which have variables like delta, gamma, and theta, I decided that I didn't like the line lengths I was getting with such variable names. I'm using ?, ?, and ? instead. It works fine, at least on my Ubuntu Linux system (and what scientist doesn't use Linux?). I also have special mathematical symbols, superscripted numbers, etc. in my program comments. It's easier to read 2x? + 3x? than 2*x**3 + 3*x**2. > > I am teaching someone Python who is having a few problems with Unicode on his Windows 7 machine. It would appear that Windows shipped with a less-than-complete Unicode font for its command shell. But that's not Python's fault. Haskell is a bit ahead of python in this respect: Prelude> let (x? , x?) = (1,2) Prelude> (x? , x?) (1,2) >>> (x? , x?) = (1,2) File "", line 1 (x? , x?) = (1,2) ^ SyntaxError: invalid character in identifier But python is ahead in another (arguably more) important aspect: Haskell gets confused by ligatures in identifiers; python gets them right >>> ?ag = 1 >>> flag 1 Prelude> let ?ag = 1 Prelude> flag :4:1: Not in scope: `flag' Hopefully python will widen its identifier-chars also From frank at chagford.com Thu Feb 12 23:22:53 2015 From: frank at chagford.com (Frank Millman) Date: Fri, 13 Feb 2015 06:22:53 +0200 Subject: Bad text appearance in IDLE References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: "ast" wrote in message news:54dc9bee$0$3046$426a34cc at news.free.fr... > Hello > > Here is how text appears in IDLE window > http://www.cjoint.com/data/0BmnEIcxVAx.htm > > Yesterday evening I had not this trouble. It appears > this morning. I restarted my computer with no effect. > > A windows Vista update has been done this morning, with about 10 fixes. I > suspect something gone wrong with this update > > Has somebody an explanation about that ? > > thx > I use Windows Server 2003. It also ran an automatic update yesterday. Something seems to have gone wrong with the system font. I don't use IDLE, but I use OutlookExpress and Textpad, and they both show a similar effect. I can live with it for now. I am hoping (with fingers crossed) that enough people will complain to Microsoft that they will issue a fix shortly. Frank Millman From tjreedy at udel.edu Fri Feb 13 00:24:35 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 13 Feb 2015 00:24:35 -0500 Subject: Python discussed in Nature In-Reply-To: <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> Message-ID: On 2/12/2015 11:07 PM, Rustom Mody wrote: > On Thursday, February 12, 2015 at 11:59:55 PM UTC+5:30, John Ladasky wrote: >> On Thursday, February 12, 2015 at 3:08:10 AM UTC-8, Fabien wrote: >> >>> ... what a coincidence then that a huge majority of scientists >>> (including me) dont care AT ALL about unicode. But since scientists are >>> not paid to rewrite old code, the scientific world is still stuck to >>> python 2. >> >> I'm a scientist. I'm a happy Python 3 user who migrated from Python 2 about two years ago. >> >> And I use Unicode in my Python. In implementing some mathematical models which have variables like delta, gamma, and theta, I decided that I didn't like the line lengths I was getting with such variable names. I'm using ?, ?, and ? instead. It works fine, at least on my Ubuntu Linux system (and what scientist doesn't use Linux?). I also have special mathematical symbols, superscripted numbers, etc. in my program comments. It's easier to read 2x? + 3x? than 2*x**3 + 3*x**2. >> >> I am teaching someone Python who is having a few problems with Unicode on his Windows 7 machine. It would appear that Windows shipped with a less-than-complete Unicode font for its command shell. But that's not Python's fault. > > Haskell is a bit ahead of python in this respect: > > Prelude> let (x? , x?) = (1,2) > Prelude> (x? , x?) > (1,2) > >>>> (x? , x?) = (1,2) > File "", line 1 > (x? , x?) = (1,2) > ^ > SyntaxError: invalid character in identifier > > But python is ahead in another (arguably more) important aspect: > Haskell gets confused by ligatures in identifiers; python gets them right > >>>> ?ag = 1 >>>> flag > 1 > > Prelude> let ?ag = 1 > Prelude> flag > > :4:1: Not in scope: `flag' > > Hopefully python will widen its identifier-chars also Python (supposedly) follows the Unicode definition based on character classes, as documented. If the Unicode definition in fact allows subscripts, then Python should also. If you want Python to broaden its definition beyond unicode, you will have to advocate and persuade. It will not 'just happen'. -- Terry Jan Reedy From wxjmfauth at gmail.com Fri Feb 13 02:02:43 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 12 Feb 2015 23:02:43 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> Message-ID: <486a3255-e3e9-4b54-8039-3f186f715c47@googlegroups.com> Le vendredi 13 f?vrier 2015 05:07:45 UTC+1, Rustom Mody a ?crit?: > > Hopefully python will widen its identifier-chars also On that side Python is doing correcty. This has been already discussed. See ??? It's time to spend some time in understanding all this stuff, Unicode, Python, Python/Unicode and the coding of characters. I have the feeling most of you have a understanding of Unicode only via Python. Unicode is "independent" from computer languages or OS. But all software has to use it. From wxjmfauth at gmail.com Fri Feb 13 02:05:53 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 12 Feb 2015 23:05:53 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: <379f5ed1-b137-43e9-8b8b-8da9ee9344a8@googlegroups.com> Le vendredi 13 f?vrier 2015 04:12:01 UTC+1, Steven D'Aprano a ?crit?: > > > Oooh! What is your font of choice for this? > I recommended many times to toy a little bit with a TeX unicode engines. It's a very nice way to learn Unicode. From hniksic at gmail.com Fri Feb 13 04:26:35 2015 From: hniksic at gmail.com (=?UTF-8?B?SHJ2b2plIE5pa8WhacSH?=) Date: Fri, 13 Feb 2015 10:26:35 +0100 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: Ian Kelly writes: > When you specify the a precision of 15 in your format string, you're > telling it to take the first 15 of those. It doesn't care that the > last couple of those are zeros, because as far as it's concerned, > those digits are significant. OK, it's a bit surprising, but also consistent with the rest of the decimal module. Thanks for clearing it up. My concrete use case is printing an arbitrary fraction as a user-readable decimal, rounded to the specified number of digits, and using the exponential notation where appropriate: import decimal _dec_fmt_context = decimal.Context(prec=15, rounding=decimal.ROUND_HALF_UP) def _format(frac): with decimal.localcontext(_dec_fmt_context): dec = decimal.Decimal(frac.numerator) / decimal.Decimal(frac.denominator) return '{:g}'.format(dec) The decimal obtained by dividing the numerator with the denominator includes trailing zeros. Calling normalize() to get rid of them will have the unfortunate side effect of turning 9806650 into 9.80665e+6, and the method recommended in the documentation: def remove_exponent(d): return d.quantize(decimal.Decimal(1)) if d == d.to_integral() else d.normalize() ...will raise "decimal.InvalidOperation: quantize result has too many digits for current context" when the number is too large. For now I'm emulating the behavior of '%g' on floats using rstrip('0') to get rid of the trailing zeros: ... s = '{:g}'.format(dec) if '.' in s and 'e' not in s: s = s.rstrip('0') s = s.rstrip('.') return s From wxjmfauth at gmail.com Fri Feb 13 04:36:04 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 13 Feb 2015 01:36:04 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <486a3255-e3e9-4b54-8039-3f186f715c47@googlegroups.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <31d8de6c-b368-4a46-a472-e77bb48bb2da@googlegroups.com> <486a3255-e3e9-4b54-8039-3f186f715c47@googlegroups.com> Message-ID: Le vendredi 13 f?vrier 2015 08:03:05 UTC+1, wxjm... at gmail.com a ?crit?: > Le vendredi 13 f?vrier 2015 05:07:45 UTC+1, Rustom Mody a ?crit?: > > > > Hopefully python will widen its identifier-chars also > > On that side Python is doing correcty. > This has been already discussed. > See ??? > > It's time to spend some time in understanding > all this stuff, Unicode, Python, Python/Unicode > and the coding of characters. > > I have the feeling most of you have a understanding > of Unicode only via Python. > > Unicode is "independent" from computer languages or OS. > But all software has to use it. Having said this I'm the first to recognize there plenty of things I'm not understanding. Eg, I never compiled/built Python on Windows. I'm not able to download/install the tools to do it. From wxjmfauth at gmail.com Fri Feb 13 05:03:30 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 13 Feb 2015 02:03:30 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> Message-ID: <82dd0631-cc2a-4488-8c86-6e14b844a446@googlegroups.com> Le jeudi 12 f?vrier 2015 19:38:44 UTC+1, Chris Angelico a ?crit?: > On Fri, Feb 13, 2015 at 5:29 AM, John Ladasky > wrote: > > It works fine, at least on my Ubuntu Linux system (and what scientist doesn't use Linux?). I also have special mathematical symbols, superscripted numbers, etc. in my program comments. It's easier to read 2x? + 3x? than 2*x**3 + 3*x**2. > > > > I am teaching someone Python who is having a few problems with Unicode on his Windows 7 machine. It would appear that Windows shipped with a less-than-complete Unicode font for its command shell. But that's not Python's fault. > > > > Yes, Windows's default terminal/console does have issues. If all your > text is staying within the BMP, you may be able to run it within IDLE > to get somewhat better results; or PowerShell may help. But as you > say, that's not Python's fault. > > Fortunately, it's not difficult to write a GUI program that > manipulates Unicode text, or something that works entirely with files > and leaves the display up to something else (maybe a good text editor, > or a web browser). All your internals are working perfectly, it's just > the human interface that's a bit harder. And only on flawed/broken > platforms. > The windows terminal may not be very friendly (and it's not broken). >From a unicode point of view, I belong to those, who finds the windows/unicode ecosystem excellent and very coherent (utf-16). From marfig at gmail.com Fri Feb 13 05:26:34 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Fri, 13 Feb 2015 10:26:34 +0000 (UTC) Subject: Dictionary from sqlite3.Row and PyCharm unresolved reference Message-ID: <1a194e0a113d8d215970f15d910@nntp.aioe.org> Currently i'm using the following code to transform a row fetched from an sqlite database into a dictionary property: def __init__(self, id_): self.id = id_ self.data = None ... conn = sqlite3.connect('data') conn.row_factory = sqlite3.Row row = conn.execute(query, {'id': str(id_)}).fetchone() conn.close() if row: self.data = dict(zip(row.keys(), tuple(row))) I have two questions: 1. Is this an acceptable idiom for the construction of self.data dictionary, or do I have a better and more readable option? 2. How can I stop PyCharm from giving me an unresolved reference warning at row.keys()? It's not a big deal, but slightly annoying. But I'd rather prefer a method that doesn't completely disable other valid cases. From priisdk at gmail.com Fri Feb 13 05:56:40 2015 From: priisdk at gmail.com (Poul Riis) Date: Fri, 13 Feb 2015 02:56:40 -0800 (PST) Subject: pyqtgraph window position Message-ID: <94334ddf-b3ad-4d5a-a11a-237b8b9f494a@googlegroups.com> I can control the size of my pyqtgraph window below with 'resize'. But how can I control the position on the screen? Poul Riis import pyqtgraph as pg w = pg.GraphicsWindow() w.resize(250,400) for i in range(4): w.addPlot(0, i) def onClick(event): but=event.button() print("but: ",but) w.scene().sigMouseClicked.connect(onClick) From __peter__ at web.de Fri Feb 13 06:33:22 2015 From: __peter__ at web.de (Peter Otten) Date: Fri, 13 Feb 2015 12:33:22 +0100 Subject: Dictionary from sqlite3.Row and PyCharm unresolved reference References: <1a194e0a113d8d215970f15d910@nntp.aioe.org> Message-ID: Mario Figueiredo wrote: > Currently i'm using the following code to transform a row fetched from an > sqlite database into a dictionary property: class Unknown: > def __init__(self, id_): > self.id = id_ > self.data = None > ... > conn = sqlite3.connect('data') > conn.row_factory = sqlite3.Row > row = conn.execute(query, {'id': str(id_)}).fetchone() > conn.close() > > if row: > self.data = dict(zip(row.keys(), tuple(row))) > > I have two questions: > > 1. Is this an acceptable idiom for the construction of self.data > dictionary, > or do I have a better and more readable option? If sqlite3.Row is not dict-like enough to use it directly self.data = row you can either convert the Row object with self.data = dict(row) or use the recipe for a row factory provided in the documentation at https://docs.python.org/dev/library/sqlite3.html#sqlite3.Connection.row_factory that constructs a dict directly. And now an unsolicited remark: if you have more than one instance of Unknown you might read the data outside the initialiser or at least keep the connection open and pass a connection or cursor object to the initialiser. From ndbecker2 at gmail.com Fri Feb 13 08:33:15 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 13 Feb 2015 08:33:15 -0500 Subject: line_profiler: what am I doing wrong? References: <54daa1d6$0$12979$c3e8da3$5496439d@news.astraweb.com> <54daada4$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: Robert Kern wrote: > On 2015-02-11 01:17, Steven D'Aprano wrote: >> Neal Becker wrote: >> >> >>> To quote from https://pypi.python.org/pypi/line_profiler/ >>> >>> $ kernprof -l script_to_profile.py >>> kernprof will create an instance of LineProfiler and insert it into the >>> __builtins__ namespace with the name profile. >> >> Ewww!!!! What a Ruby-esque interface, that makes me sad :-( > > This is not a production library. It's a development tool designed to > help developers shorten the cycle time for investigating these kinds of > issues. Well, *a* developer; i.e. me. If it helps anyone else, yahtzee! > >> And what if you >> have your own profile global name? > > Then you can pull it out from __builtin__ with a different name and use that > other name. > >> And *wrong* too. `__builtins__` is a private CPython implementation detail. >> The way to monkey-patch the built-ins in Python 2 is to inject the object >> into `__builtin__` (no s), or `builtins` in Python 3. > > And indeed that is how it is implemented. Referring to that namespace as the > "`__builtins__` namespace" isn't *wrong*. It may mislead you into thinking > I've implemented it one particular way, if you are desperate to find a nit to > pick. > >> Seeing as >> line_profiler is written in C, perhaps the author (Robert Kern) doesn't >> care about supporting Jython or IronPython, but there may be Python >> implementations (PyPy perhaps?) which can run C code but don't have >> __builtins__. > > Indeed, I do not care about any of them. PyPy does not implement CPython's > tracing API: > > https://bitbucket.org/pypy/pypy/src/2b2163d65ee437646194a1ceb2a3153db24c5f7e/pypy/module/cpyext/stubs.py?at=default#cl-1286 > Hi Robert, any idea why line_profiler is not working? I've used it fine in the past. -- -- Those who don't understand recursion are doomed to repeat it From ndbecker2 at gmail.com Fri Feb 13 08:35:13 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 13 Feb 2015 08:35:13 -0500 Subject: line_profiler: what am I doing wrong? References: Message-ID: Robert Kern wrote: > @profile > def run(): > pass > > run() No, this doesn't work either. Same failure kernprof -l test_prof.py Wrote profile results to test_prof.py.lprof Traceback (most recent call last): File "/home/nbecker/.local/bin/kernprof", line 9, in load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, in main execfile(script_file, ns, ns) File "test_prof.py", line 1, in @profile NameError: name 'profile' is not defined -- -- Those who don't understand recursion are doomed to repeat it From invalid at invalid.invalid Fri Feb 13 09:02:03 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 13 Feb 2015 14:02:03 +0000 (UTC) Subject: Floating point "g" format not stripping trailing zeros References: Message-ID: On 2015-02-12, Ian Kelly wrote: > On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: > >> {:.15g} is supposed to give 15 digits of precision, but with trailing >> zeros removed. > > The doc says with "insignificant" trailing zeros removed, not all > trailing zeros. Can somebody explain the difference between "significant" and "insignificant" tailing zeros to somebody who barely passed his single numerical methods class? [Though I have, on occasion, had to tinker with the innards of SW floating point libraries and could fish a hardcopy of IEEE-754 out of a filing cabinet if needed.] -- Grant Edwards grant.b.edwards Yow! My polyvinyl cowboy at wallet was made in Hong gmail.com Kong by Montgomery Clift! From nomail at invalid.com Fri Feb 13 10:04:33 2015 From: nomail at invalid.com (ast) Date: Fri, 13 Feb 2015 16:04:33 +0100 Subject: Bad text appearance in IDLE In-Reply-To: References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: <54de1285$0$2980$426a74cc@news.free.fr> "Terry Reedy" a ?crit dans le message de news:mailman.18714.1423775571.18130.python-list at python.org... > Do you get anything similar when running the console interpreter? yes > import tkinter as tk > > root = tk.Tk() > text = tk.Text() > text.pack() > text.insert('1.0', 'this is a test eeeccee') > same problem with the text 'this is a test eeeccee' written on the Text() widget From nomail at invalid.com Fri Feb 13 10:06:09 2015 From: nomail at invalid.com (ast) Date: Fri, 13 Feb 2015 16:06:09 +0100 Subject: Bad text appearance in IDLE In-Reply-To: References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: <54de12e4$0$3072$426a74cc@news.free.fr> "Frank Millman" a ?crit dans le message de news:mailman.18720.1423801380.18130.python-list at python.org... > I use Windows Server 2003. It also ran an automatic update yesterday. Something seems to have gone > wrong with the system font. I don't use IDLE, but I use OutlookExpress and Textpad, and they both > show a similar effect. > > I can live with it for now. me too > I am hoping (with fingers crossed) that enough people will complain to Microsoft that they will > issue a fix shortly. > thx for informations. so I am going to complain to microsoft too From russell at infiniteio.com Fri Feb 13 10:39:05 2015 From: russell at infiniteio.com (Russell) Date: Fri, 13 Feb 2015 07:39:05 -0800 (PST) Subject: ctypes.cdll.LoadLibrary() freezes when loading a .so that contains dlopen() Message-ID: <8f93337c-164f-4bcb-aa57-a3aa8a33b38b@googlegroups.com> I have a shared library, libfoo.so, that references another .so which isn't linked but instead loaded at runtime with myso=dlopen("/usr/local/lib/libbar.so", RTLD_NOW); when I try to load it with ctypes, the call hangs and I have to ctl-c. (build)[dev]$ export LD_LIBRARY_PATH=/usr/local/bin (build)[dev]$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes >>> ctypes.cdll.LoadLibrary ('/usr/local/lib/libfoo.so') <--- This call hangs and have to ctl-C ^CTraceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary return self._dlltype(name) File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__ self._handle = _dlopen(self._name, mode) KeyboardInterrupt >>> My first thought was that It couldn't find libbar.so, but if I remove that file, python seg faults: /usr/local/lib/libbar.so: cannot open shared object file: No such file or directorySegmentation fault (core dumped), so it appears that it finds the dlopen() file but freezes waiting for ??? This is on ubuntu 14.4 server. C code is compiled with -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC I also get the same reaction on python3.4 Thanks in advance From ginolampascione at gmx.com Fri Feb 13 10:46:22 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Fri, 13 Feb 2015 07:46:22 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <2c61860e-d9c3-4fd1-8f22-68ceff90788d@googlegroups.com> -- redacted -- From ginolampascione at gmx.com Fri Feb 13 10:48:33 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Fri, 13 Feb 2015 07:48:33 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <8c738b51-e2ea-49bf-b13c-de3c5a19bcec@googlegroups.com> -- redacted -- From ginolampascione at gmx.com Fri Feb 13 10:52:00 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Fri, 13 Feb 2015 07:52:00 -0800 (PST) Subject: -- redacted -- In-Reply-To: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> References: <2635cb3f-446a-45dc-8313-3178adfd24af@googlegroups.com> Message-ID: <659dc4dd-0b59-46aa-a1ca-37a5c7c87d6a@googlegroups.com> -- redacted -- From wxjmfauth at gmail.com Fri Feb 13 11:46:07 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 13 Feb 2015 08:46:07 -0800 (PST) Subject: Wildly OT: pop-up virtual keyboard for Mac or Linux? In-Reply-To: References: Message-ID: Le mercredi 11 f?vrier 2015 14:03:53 UTC+1, Tim Chase a ?crit?: > On 2015-02-10 19:37, Ned Deily wrote: > > On OS X, the system provides both a "Character Viewer" (which > > allows the selection of any Unicode character > > Windows also provides charmap.exe which provides similar > functionality, though last I checked it, it still had the feel of a > Win3.1 app (usability was atrocious) > > -tkc It is working very well and it is very simple to use. On top of this, on my win box, the names of the characters are presented like this: >>> >>> fourbiunicode.FrenchIsoName(ord('\u0178')) 'LETTRE MAJUSCULE LATINE Y TR?MA' >>> fourbiunicode.FrenchIsoName(ord('?')) 'LETTRE MAJUSCULE LATINE Y TR?MA' >>> Note the correct non-ascii name in French. jmf From ian.g.kelly at gmail.com Fri Feb 13 11:52:10 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 09:52:10 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 2:26 AM, Hrvoje Nik?i? wrote: > Ian Kelly writes: >> When you specify the a precision of 15 in your format string, you're >> telling it to take the first 15 of those. It doesn't care that the >> last couple of those are zeros, because as far as it's concerned, >> those digits are significant. > > OK, it's a bit surprising, but also consistent with the rest of the > decimal module. Thanks for clearing it up. > > My concrete use case is printing an arbitrary fraction as a > user-readable decimal, rounded to the specified number of digits, and > using the exponential notation where appropriate: > > import decimal > _dec_fmt_context = decimal.Context(prec=15, rounding=decimal.ROUND_HALF_UP) > def _format(frac): > with decimal.localcontext(_dec_fmt_context): > dec = decimal.Decimal(frac.numerator) / > decimal.Decimal(frac.denominator) > return '{:g}'.format(dec) > > The decimal obtained by dividing the numerator with the denominator > includes trailing zeros. Calling normalize() to get rid of them will > have the unfortunate side effect of turning 9806650 into 9.80665e+6, > and the method recommended in the documentation: > > def remove_exponent(d): > return d.quantize(decimal.Decimal(1)) if d == d.to_integral() else > d.normalize() > > ...will raise "decimal.InvalidOperation: quantize result has too many > digits for current context" when the number is too large. In that case I think you just want to normalize instead. So: >>> def remove_exponent(d): ... if d == d.to_integral(): ... try: ... return d.quantize(1) ... except decimal.InvalidOperation: ... pass ... return d.normalize() ... >>> remove_exponent(D('123000000')) Decimal('123000000') >>> remove_exponent(D('123.4567')) Decimal('123.4567') >>> remove_exponent(D('123.4567890')) Decimal('123.456789') >>> remove_exponent(D('123e6')) Decimal('123000000') >>> remove_exponent(D('1234567890123456789012345678901234567890')) Decimal('1.234567890123456789012345679E+39') From ian.g.kelly at gmail.com Fri Feb 13 11:56:28 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 09:56:28 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards wrote: > On 2015-02-12, Ian Kelly wrote: >> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: >> >>> {:.15g} is supposed to give 15 digits of precision, but with trailing >>> zeros removed. >> >> The doc says with "insignificant" trailing zeros removed, not all >> trailing zeros. > > Can somebody explain the difference between "significant" and > "insignificant" tailing zeros to somebody who barely passed his single > numerical methods class? [Though I have, on occasion, had to tinker > with the innards of SW floating point libraries and could fish a > hardcopy of IEEE-754 out of a filing cabinet if needed.] Significant digits are within the precision of the calculation. Writing 1.230 indicates that the fourth digit is known to be zero. Writing 1.23 outside a context of exact calculation indicates that the fourth digit is unknown due to insufficient precision. From marfig at gmail.com Fri Feb 13 13:46:04 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Fri, 13 Feb 2015 19:46:04 +0100 Subject: Dictionary from sqlite3.Row and PyCharm unresolved reference References: <1a194e0a113d8d215970f15d910@nntp.aioe.org> Message-ID: In article , __peter__ at web.de says... > > self.data = dict(row) I didn't realize from the documentation it could be this simple. Thanks. > > And now an unsolicited remark: if you have more than one instance of Unknown > you might read the data outside the initialiser or at least keep the > connection open and pass a connection or cursor object to the initialiser. Yes, thank you. Later I am moving the code to a factory. From john_ladasky at sbcglobal.net Fri Feb 13 14:40:10 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Fri, 13 Feb 2015 11:40:10 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thursday, February 12, 2015 at 7:12:01 PM UTC-8, Steven D'Aprano wrote: > John Ladasky wrote: > > > And I use Unicode in my Python. In implementing some mathematical models > > which have variables like delta, gamma, and theta, I decided that I didn't > > like the line lengths I was getting with such variable names. I'm using > > ?, ?, and ? instead. It works fine, at least on my Ubuntu Linux system > > (and what scientist doesn't use Linux?). I also have special mathematical > > symbols, superscripted numbers, etc. in my program comments. It's easier > > to read 2x? + 3x? than 2*x**3 + 3*x**2. > > > Oooh! What is your font of choice for this? Steven, I'm trying to answer your question. It's proving to be harder than I expected. The default font that the Geany program editor uses on my Ubuntu system renders everything I've tried. When I look up that font in Geany's Preferences menu, it is called, simply, "monospace". However, when I browse through fonts using the Ubuntu (14.04, 64-bit) Font Viewer, I do not find a font with that simple name. I find several fonts whose name includes "mono" in the string: Liberation Mono, Nimbus Mono, Ubuntu Mono, etc. All these fonts are listed as alternate choices in Geany's font selection menu. Any Unicode that I use in my Python also renders nicely in the Ubuntu terminal. In the Terminal Profile Preferences, the "use the system fixed width font" check box is selected. So, it would appear that I'm using a font that the OS is selecting for me. Which font that is, I'm still trying to figure out. From invalid at invalid.invalid Fri Feb 13 15:33:16 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 13 Feb 2015 20:33:16 +0000 (UTC) Subject: Floating point "g" format not stripping trailing zeros References: Message-ID: On 2015-02-13, Ian Kelly wrote: > On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards wrote: >> On 2015-02-12, Ian Kelly wrote: >>> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: >>> >>>> {:.15g} is supposed to give 15 digits of precision, but with trailing >>>> zeros removed. >>> >>> The doc says with "insignificant" trailing zeros removed, not all >>> trailing zeros. >> >> Can somebody explain the difference between "significant" and >> "insignificant" tailing zeros to somebody who barely passed his single >> numerical methods class? [Though I have, on occasion, had to tinker >> with the innards of SW floating point libraries and could fish a >> hardcopy of IEEE-754 out of a filing cabinet if needed.] > > Significant digits are within the precision of the calculation. > Writing 1.230 indicates that the fourth digit is known to be zero. > Writing 1.23 outside a context of exact calculation indicates that the > fourth digit is unknown due to insufficient precision. I knew that, but I was asking in the context of float/decimal's formatting function. I didn't realize that float and/or decimal had a "significant digit" property, and therefore possess significant vs. insignificant trailing zeros when represented in base-10. -- Grant Edwards grant.b.edwards Yow! Mr and Mrs PED, can I at borrow 26.7% of the RAYON gmail.com TEXTILE production of the INDONESIAN archipelago? From ian.g.kelly at gmail.com Fri Feb 13 16:01:11 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 14:01:11 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 1:33 PM, Grant Edwards wrote: > On 2015-02-13, Ian Kelly wrote: >> On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards wrote: >>> On 2015-02-12, Ian Kelly wrote: >>>> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: >>>> >>>>> {:.15g} is supposed to give 15 digits of precision, but with trailing >>>>> zeros removed. >>>> >>>> The doc says with "insignificant" trailing zeros removed, not all >>>> trailing zeros. >>> >>> Can somebody explain the difference between "significant" and >>> "insignificant" tailing zeros to somebody who barely passed his single >>> numerical methods class? [Though I have, on occasion, had to tinker >>> with the innards of SW floating point libraries and could fish a >>> hardcopy of IEEE-754 out of a filing cabinet if needed.] >> >> Significant digits are within the precision of the calculation. >> Writing 1.230 indicates that the fourth digit is known to be zero. >> Writing 1.23 outside a context of exact calculation indicates that the >> fourth digit is unknown due to insufficient precision. > > I knew that, but I was asking in the context of float/decimal's > formatting function. I didn't realize that float and/or decimal had a > "significant digit" property, and therefore possess significant vs. > insignificant trailing zeros when represented in base-10. A Decimal object consists of a sign, a tuple of decimal digits, and an exponent. If the digits are, e.g, (1, 2, 3, 0, 0, 0), then that would be equal to the Decimal with the digits (1, 2, 3) and the same sign and exponent, but the explicit presence of the trailing zeros indicates their significance. If this doesn't answer your question, then I'm not really sure what you're asking. From davea at davea.name Fri Feb 13 16:15:08 2015 From: davea at davea.name (Dave Angel) Date: Fri, 13 Feb 2015 16:15:08 -0500 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: <54DE695C.1090103@davea.name> On 02/13/2015 03:33 PM, Grant Edwards wrote: > On 2015-02-13, Ian Kelly wrote: >> Significant digits are within the precision of the calculation. >> Writing 1.230 indicates that the fourth digit is known to be zero. >> Writing 1.23 outside a context of exact calculation indicates that the >> fourth digit is unknown due to insufficient precision. > > I knew that, but I was asking in the context of float/decimal's > formatting function. I didn't realize that float and/or decimal had a > "significant digit" property, and therefore possess significant vs. > insignificant trailing zeros when represented in base-10. > Just jumping in here, and somebody please correct me if I mess it up. Built-in binary floating point (float) has a fixed number of bits for mantissa, and separate bits for exponent and sign. Because of those fixed number of bits, no assumption can be made as to how many of them are relevant. On the other hand, the Decimal package has a way that the programmer can tell how many digits to use at each stage of the calculation. So if the programmer bothered to set it to the correct precision, the print logic (could) use that to decide about trailing zeroes. I have no idea whether it does, but this thread would seem to say it does. I also have no definite opinion as to whether that's reasonable, or whether most calculations are done by setting digits to about twice what's needed, and rounding at the end. I know I did exactly that when I wrote a variable length math package to double-check the accuracy of a fixed precision package I was developing, 40 years ago (long before the IEEE-754 standard began meetings). The fixed precision package was fast, and used lots of clever(?) shortcuts for speed. The variable one was written very brute force, ran maybe 100 times slower (on another machine), but the results could be compared with automatic algorithms. For simple arithmetic, not too big a deal, but for transcendentals, the error analysis was very important. For example, the fast algorithm was a custom chebyshev, while the reference implementation was Taylor series. -- DaveA From invalid at invalid.invalid Fri Feb 13 16:22:40 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 13 Feb 2015 21:22:40 +0000 (UTC) Subject: Floating point "g" format not stripping trailing zeros References: Message-ID: On 2015-02-13, Dave Angel wrote: > On 02/13/2015 03:33 PM, Grant Edwards wrote: >> On 2015-02-13, Ian Kelly wrote: >> >>> Significant digits are within the precision of the calculation. >>> Writing 1.230 indicates that the fourth digit is known to be zero. >>> Writing 1.23 outside a context of exact calculation indicates that the >>> fourth digit is unknown due to insufficient precision. >> >> I knew that, but I was asking in the context of float/decimal's >> formatting function. I didn't realize that float and/or decimal had a >> "significant digit" property, and therefore possess significant vs. >> insignificant trailing zeros when represented in base-10. > > Just jumping in here, and somebody please correct me if I mess it up. > > Built-in binary floating point (float) has a fixed number of bits for > mantissa, and separate bits for exponent and sign. Because of those > fixed number of bits, no assumption can be made as to how many of them > are relevant. Right. > On the other hand, the Decimal package has a way that the programmer > can tell how many digits to use at each stage of the calculation. That's what surpised me. From TFM: https://docs.python.org/2/library/decimal.html: * The decimal module incorporates a notion of significant places so that 1.30 + 1.20 is 2.50. The trailing zero is kept to indicate significance. This is the customary presentation for monetary applications. For multiplication, the ?schoolbook? approach uses all the figures in the multiplicands. For instance, 1.3 * 1.2 gives 1.56 while 1.30 * 1.20 gives 1.5600. > So if the programmer bothered to set it to the correct precision, the > print logic (could) use that to decide about trailing zeroes. I have > no idea whether it does, but this thread would seem to say it does. It seems to. Pretty cool. -- Grant Edwards grant.b.edwards Yow! LOOK!! Sullen at American teens wearing gmail.com MADRAS shorts and "Flock of Seagulls" HAIRCUTS! From ian.g.kelly at gmail.com Fri Feb 13 16:40:01 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 14:40:01 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 2:22 PM, Grant Edwards wrote: > On 2015-02-13, Dave Angel wrote: >> On the other hand, the Decimal package has a way that the programmer >> can tell how many digits to use at each stage of the calculation. > > That's what surpised me. From TFM: > > https://docs.python.org/2/library/decimal.html: > > * The decimal module incorporates a notion of significant places so that > 1.30 + 1.20 is 2.50. The trailing zero is kept to indicate > significance. This is the customary presentation for monetary > applications. For multiplication, the ?schoolbook? approach uses > all the figures in the multiplicands. For instance, 1.3 * 1.2 gives > 1.56 while 1.30 * 1.20 gives 1.5600. Huh. That approach for multiplication is definitely not what I was taught in school. I was taught that the number of significant digits in the product is the lesser of the number of significant digits in either of the measured multiplicands. So 1.30 * 1.20 would be 1.56, while 1.3 * 1.2 would just be 1.6. Wikipedia appears to agree with me: http://en.wikipedia.org/wiki/Significance_arithmetic#Multiplication_and_division_using_significance_arithmetic Moreover: >>> D('1.304') * D('1.204') Decimal('1.570016') >>> D('1.295') * D('1.195') Decimal('1.547525') So 1.30 * 1.20 could be written approximately as 1.56 ? 0.01. Given that, I don't understand how the trailing zeros in 1.5600 could possibly be considered significant. From ian.g.kelly at gmail.com Fri Feb 13 16:49:13 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 14:49:13 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 2:40 PM, Ian Kelly wrote: > On Fri, Feb 13, 2015 at 2:22 PM, Grant Edwards wrote: >> On 2015-02-13, Dave Angel wrote: >>> On the other hand, the Decimal package has a way that the programmer >>> can tell how many digits to use at each stage of the calculation. >> >> That's what surpised me. From TFM: >> >> https://docs.python.org/2/library/decimal.html: >> >> * The decimal module incorporates a notion of significant places so that >> 1.30 + 1.20 is 2.50. The trailing zero is kept to indicate >> significance. This is the customary presentation for monetary >> applications. For multiplication, the ?schoolbook? approach uses >> all the figures in the multiplicands. For instance, 1.3 * 1.2 gives >> 1.56 while 1.30 * 1.20 gives 1.5600. > > Huh. That approach for multiplication is definitely not what I was > taught in school. I was taught that the number of significant digits > in the product is the lesser of the number of significant digits in > either of the measured multiplicands. So 1.30 * 1.20 would be 1.56, > while 1.3 * 1.2 would just be 1.6. Wikipedia appears to agree with me: > > http://en.wikipedia.org/wiki/Significance_arithmetic#Multiplication_and_division_using_significance_arithmetic > > Moreover: > >>>> D('1.304') * D('1.204') > Decimal('1.570016') >>>> D('1.295') * D('1.195') > Decimal('1.547525') > > So 1.30 * 1.20 could be written approximately as 1.56 ? 0.01. Given > that, I don't understand how the trailing zeros in 1.5600 could > possibly be considered significant. I guess the point here is that the paragraph isn't really talking about significance arithmetic; it's explaining how it decides how many digits to keep in the result. It may be fine for 1.30 * 1.20 to return 1.56, but it would be very confusing if 1.35 * 1.25 returned 1.69 instead of 1.6875. The wording of the paragraph seems misleading, though. From rosuav at gmail.com Fri Feb 13 17:45:55 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Feb 2015 09:45:55 +1100 Subject: Python discussed in Nature In-Reply-To: References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 14, 2015 at 6:40 AM, John Ladasky wrote: > The default font that the Geany program editor uses on my Ubuntu system renders everything I've tried. When I look up that font in Geany's Preferences menu, it is called, simply, "monospace". > That's a font alias. Unfortunately, I've never yet figured out a straight-forward way to snap the pointer; the best I can suggest is to go through your concrete font names and find one that looks the same. I use the same "Monospace" font for my MUD client's default. It's a good solid default, and I can guarantee that it "exists" on all systems, even if specific other fonts might not (especially given that I support three popular OSes). On my Debian system, it looks like "Monospace" == "DejaVu Sans Mono", so you could try looking at that. ChrisA From sturla.molden at gmail.com Fri Feb 13 17:58:37 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Fri, 13 Feb 2015 23:58:37 +0100 Subject: Python discussed in Nature In-Reply-To: <87twyrchxi.fsf@elektro.pacujo.net> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> Message-ID: On 12/02/15 15:39, Marko Rauhamaa wrote: > I write both Py2 and Py3 code, but I keep the two worlds hermetically > separated from each other. In SciPy world we run the same code on Python 2 and Python 3. Sturla From jsh at jsh.name Fri Feb 13 18:05:54 2015 From: jsh at jsh.name (Jonathan Hayward) Date: Fri, 13 Feb 2015 17:05:54 -0600 Subject: Python implementations in JavaScript Message-ID: What is the relative maturity of different Python implementations in JavaScript? Are any of the implementations ready to rely on? -- [image: Christos Jonathan Seth Hayward] Jonathan S. Hayward, a full stack web developer with Python/Django and AngularJS/jQuery. Articles on Technology ? Book: *Django JavaScript Integration* ? *Email * ? Github ? LinkedIn ? Main Website ? *Professional Site * I invite you to visit my professional site! -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Fri Feb 13 18:05:58 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 13 Feb 2015 23:05:58 +0000 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On 12/02/2015 23:46, Ian Kelly wrote: > On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nik?i? wrote: >>>>>>>> from decimal import Decimal as D >>>>>>>> x = D(1)/D(999) >>>>>>>> '{:.15g}'.format(x) >>>>> >>>>> '0.00100100100100100' >> [...] >>>> I'd say it's a bug. P is 15, you've got 17 digits after the decimal place >>>> and two of those are insignificant trailing zeros. >>> >>> Actually it's the float version that doesn't match the documentation. >>> In the decimal version, sure there are 17 digits after the decimal >>> place there, but the first two -- which are leading zeroes -- would >>> not normally be considered significant. >> >> {:.15g} is supposed to give 15 digits of precision, but with trailing >> zeros removed. > > The doc says with "insignificant" trailing zeros removed, not all > trailing zeros. > >> For example, '{:.15g}'.format(Decimal('0.5')) should >> yield '0.5', not '0.500000000000000' -- and, it indeed does. It is >> only for some numbers that trailing zeros are not removed, which looks >> like a bug. The behavior of floats matches both the documentation and >> other languages using the 'g' decimal format, such as C. > > Ah, I see now what's going on here. With floats, there is really no > notion of significant digits. The values 0.5 and 0.50000 are > completely equivalent. With decimals, that's not exactly true; if you > give the decimal a trailing zero then you are telling it that the zero > is significant. > >>>> Decimal('0.5') > Decimal('0.5') >>>> Decimal('0.50000') > Decimal('0.50000') >>>> Decimal('0.5').as_tuple() > DecimalTuple(sign=0, digits=(5,), exponent=-1) >>>> Decimal('0.50000').as_tuple() > DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5) > > These are distinct; the decimal knows how many significant digits you > passed it. As a result, these are also distinct: > >>>> '{:.4g}'.format(Decimal('0.5')) > '0.5' >>>> '{:.4g}'.format(Decimal('0.50000')) > '0.5000' > > Now what happens in your original example of 1/999? The default > decimal context uses 28 digits of precision, so the result of that > calculation will have 28 significant digits in it. > >>>> decimal.getcontext().prec > 28 >>>> Decimal(1) / Decimal(999) > Decimal('0.001001001001001001001001001001') > > When you specify the a precision of 15 in your format string, you're > telling it to take the first 15 of those. It doesn't care that the > last couple of those are zeros, because as far as it's concerned, > those digits are significant. > I still think it's a bug as the 'p' being referred to in the OP's original message is "The precision is a decimal number indicating how many digits should be displayed after the decimal point for a floating point value formatted with 'f' and 'F', or before and after the decimal point for a floating point value formatted with 'g' or 'G'". In other words is has nothing to do with the precision of the underlying number. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Fri Feb 13 19:11:01 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 17:11:01 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 4:05 PM, Mark Lawrence wrote: > I still think it's a bug as the 'p' being referred to in the OP's original > message is "The precision is a decimal number indicating how many digits > should be displayed after the decimal point for a floating point value > formatted with 'f' and 'F', or before and after the decimal point for a > floating point value formatted with 'g' or 'G'". In other words is has > nothing to do with the precision of the underlying number. I read that paragraph as describing generally what the precision means, with the table below detailing more specifically how it is used. I think for 'g' this is just trying to contrast it with 'f' where it really is just the number of digits displayed after the decimal point. The table entry for 'g' on the other hand quite clearly says, "for a given precision p >= 1, this rounds the number to p significant digits". From breamoreboy at yahoo.co.uk Fri Feb 13 20:22:24 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 14 Feb 2015 01:22:24 +0000 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On 14/02/2015 00:11, Ian Kelly wrote: > On Fri, Feb 13, 2015 at 4:05 PM, Mark Lawrence wrote: >> I still think it's a bug as the 'p' being referred to in the OP's original >> message is "The precision is a decimal number indicating how many digits >> should be displayed after the decimal point for a floating point value >> formatted with 'f' and 'F', or before and after the decimal point for a >> floating point value formatted with 'g' or 'G'". In other words is has >> nothing to do with the precision of the underlying number. > > I read that paragraph as describing generally what the precision > means, with the table below detailing more specifically how it is > used. I think for 'g' this is just trying to contrast it with 'f' > where it really is just the number of digits displayed after the > decimal point. The table entry for 'g' on the other hand quite clearly > says, "for a given precision p >= 1, this rounds the number to p > significant digits". > With insignificant trailing zeros removed. So if I'm asking for 15 significant digits *in the output* I don't expect to see those zeros as I don't see them as significant. If I did I'd have p set higher. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Fri Feb 13 20:26:50 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 13 Feb 2015 18:26:50 -0700 Subject: ctypes.cdll.LoadLibrary() freezes when loading a .so that contains dlopen() In-Reply-To: <8f93337c-164f-4bcb-aa57-a3aa8a33b38b@googlegroups.com> References: <8f93337c-164f-4bcb-aa57-a3aa8a33b38b@googlegroups.com> Message-ID: On Fri, Feb 13, 2015 at 8:39 AM, Russell wrote: > I have a shared library, libfoo.so, that references another .so which isn't linked but instead loaded at runtime with myso=dlopen("/usr/local/lib/libbar.so", RTLD_NOW); when I try to load it with ctypes, the call hangs and I have to ctl-c. > > (build)[dev]$ export LD_LIBRARY_PATH=/usr/local/bin > (build)[dev]$ python > Python 2.7.6 (default, Mar 22 2014, 22:59:56) > [GCC 4.8.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import ctypes >>>> ctypes.cdll.LoadLibrary ('/usr/local/lib/libfoo.so') <--- This call hangs and have to ctl-C > ^CTraceback (most recent call last): > File "", line 1, in > File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary > return self._dlltype(name) > File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__ > self._handle = _dlopen(self._name, mode) > KeyboardInterrupt >>>> > > > My first thought was that It couldn't find libbar.so, but if I remove that file, python seg faults: /usr/local/lib/libbar.so: cannot open shared object file: No such file or directorySegmentation fault (core dumped), so it appears that it finds the dlopen() file but freezes waiting for ??? > > This is on ubuntu 14.4 server. C code is compiled with -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC This seems to work for me (Mint 17). $ cat a.c #include #include void _init() { printf("a.so._init\n"); dlopen("./b.so", RTLD_NOW); } $ cat b.c #include void _init() { printf("b.so._init\n"); } $ gcc -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC -shared -nostartfiles -o a.so a.c $ gcc -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC -shared -nostartfiles -o b.so b.c $ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes >>> ctypes.cdll.LoadLibrary('./a.so') a.so._init b.so._init Can you boil down the issue that you're seeing into a minimal reproducible example? From cousinstanley at gmail.com Fri Feb 13 20:47:55 2015 From: cousinstanley at gmail.com (Cousin Stanley) Date: Fri, 13 Feb 2015 18:47:55 -0700 Subject: pyqtgraph window position References: <94334ddf-b3ad-4d5a-a11a-237b8b9f494a@googlegroups.com> Message-ID: > I can control the size of my pyqtgraph window below > with 'resize'. > > But how can I control the position on the screen ? > > import pyqtgraph as pg > > w = pg.GraphicsWindow() > w.resize(250,400) > > for i in range(4): > w.addPlot(0, i) > > def onClick(event): > but=event.button() > print("but: ",but) > > w.scene().sigMouseClicked.connect(onClick) try .... pos_x = 300 pos_y = 400 w.move( pos_x , pos_y ) -- Stanley C. Kitching Human Being Phoenix, Arizona From greg.ewing at canterbury.ac.nz Fri Feb 13 22:22:57 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 14 Feb 2015 16:22:57 +1300 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: Mark Lawrence wrote: > I still think it's a bug as the 'p' being referred to in the OP's > original message is "The precision is a decimal number indicating how > many digits should be displayed after the decimal point for a floating > point value formatted with 'f' and 'F', or before and after the decimal > point for a floating point value formatted with 'g' or 'G'". In other > words is has nothing to do with the precision of the underlying number. But it does say that "insignificant trailing zeros are removed", not just "trailing zeros are removed". However, it's possible that "insignificant" was just intended to mean "after the decimal point". The specification is a bit vague. -- Greg From russell at infiniteio.com Fri Feb 13 23:49:11 2015 From: russell at infiniteio.com (Russell) Date: Fri, 13 Feb 2015 20:49:11 -0800 (PST) Subject: ctypes.cdll.LoadLibrary() freezes when loading a .so that contains dlopen() In-Reply-To: References: <8f93337c-164f-4bcb-aa57-a3aa8a33b38b@googlegroups.com> Message-ID: <2c3d5c8c-40ba-4a7b-9ec1-4d95b525a644@googlegroups.com> On Friday, February 13, 2015 at 7:27:54 PM UTC-6, Ian wrote: > On Fri, Feb 13, 2015 at 8:39 AM, Russell wrote: > > I have a shared library, libfoo.so, that references another .so which isn't linked but instead loaded at runtime with myso=dlopen("/usr/local/lib/libbar.so", RTLD_NOW); when I try to load it with ctypes, the call hangs and I have to ctl-c. > > > > (build)[dev]$ export LD_LIBRARY_PATH=/usr/local/bin > > (build)[dev]$ python > > Python 2.7.6 (default, Mar 22 2014, 22:59:56) > > [GCC 4.8.2] on linux2 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> import ctypes > >>>> ctypes.cdll.LoadLibrary ('/usr/local/lib/libfoo.so') <--- This call hangs and have to ctl-C > > ^CTraceback (most recent call last): > > File "", line 1, in > > File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary > > return self._dlltype(name) > > File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__ > > self._handle = _dlopen(self._name, mode) > > KeyboardInterrupt > >>>> > > > > > > My first thought was that It couldn't find libbar.so, but if I remove that file, python seg faults: /usr/local/lib/libbar.so: cannot open shared object file: No such file or directorySegmentation fault (core dumped), so it appears that it finds the dlopen() file but freezes waiting for ??? > > > > This is on ubuntu 14.4 server. C code is compiled with -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC > > This seems to work for me (Mint 17). > > > $ cat a.c > #include > #include > > void _init() { > printf("a.so._init\n"); > dlopen("./b.so", RTLD_NOW); > } > $ cat b.c > #include > > void _init() { > printf("b.so._init\n"); > } > $ gcc -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC > -shared -nostartfiles -o a.so a.c > $ gcc -std=gnu11 -Wall -Werror -m64 -march=x86-64 -mavx -g -fPIC > -shared -nostartfiles -o b.so b.c > $ python > Python 2.7.6 (default, Mar 22 2014, 22:59:56) > [GCC 4.8.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import ctypes > >>> ctypes.cdll.LoadLibrary('./a.so') > a.so._init > b.so._init > > > > Can you boil down the issue that you're seeing into a minimal > reproducible example? Thanks Ian, that's a good way to examine my problem. Your code worked for me also. I've taken your code and expanded it to be more representative of my issue. One twist that i found was that I'm calling back into a.so from b.so. I've captured that in the following code. Note that this works for me without error!! There must be something else in my libs, that is causing this. I'm still working to find my issue but I'm just replying to thank you and give anyone an update :) (build)[w]$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes >>> ctypes.cdll.LoadLibrary('./a.so') a.so._init b.so._init b.so recived: called from a.so in a_function >>> [w]$ cat a.c #include #include #include void (*mymessage)(char *message); void *myso; __attribute__((constructor)) void Init(void) { printf("a.so._init\n"); char* error; myso = dlopen("./b.so", RTLD_NOW); if (!myso) { fputs (dlerror(), stderr); exit(1); } mymessage = dlsym(myso, "printMessage"); if ((error = dlerror()) != NULL) { fputs(error, stderr); exit(1); } mymessage("called from a.so"); } void a_function(void) { printf("in a_function\n"); } __attribute__((destructor)) void Close(void) { dlclose(myso); } [w]$ cat b.c #include extern void a_function(void); __attribute__((constructor)) void Init(void) { printf("b.so._init\n"); } void printMessage(char *message) { printf("b.so recived: %s\n", message); a_function(); } __attribute__((destructor)) void Close(void) { } [w]$ cat build /usr/bin/gcc -o ./a.os -c -std=gnu11 -Wall -Werror -Wa,-ahl=./a.os.s -m64 -march=x86-64 -mavx -g -DDEBUG -fPIC a.c /usr/bin/gcc -o ./a.so -Wl,--export-dynamic -shared -Wl,-rpath=./ -ldl ./a.os /usr/bin/gcc -o ./b.os -c -std=gnu11 -Wall -Werror -Wa,-ahl=./b.os.s -m64 -march=x86-64 -mavx -g -DDEBUG -fPIC b.c /usr/bin/gcc -o ./b.so -Wl,--export-dynamic -shared -Wl,-rpath=./ b.os a.so From cousinstanley at gmail.com Sat Feb 14 00:59:01 2015 From: cousinstanley at gmail.com (Cousin Stanley) Date: Fri, 13 Feb 2015 22:59:01 -0700 Subject: pyqtgraph window position References: <94334ddf-b3ad-4d5a-a11a-237b8b9f494a@googlegroups.com> Message-ID: > I can control the size of my pyqtgraph window below with 'resize'. > > But how can I control the position on the screen? > .... Also, try .... w.setGeometry( x_pos , y_pos , width , height ) -- Stanley C. Kitching Human Being Phoenix, Arizona From wxjmfauth at gmail.com Sat Feb 14 03:39:09 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Sat, 14 Feb 2015 00:39:09 -0800 (PST) Subject: Python discussed in Nature In-Reply-To: References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <87zj8jcqxr.fsf@elektro.pacujo.net> <87twyrchxi.fsf@elektro.pacujo.net> Message-ID: <946f0a3c-951c-4bd2-bfe0-81c8c7c74b47@googlegroups.com> Le vendredi 13 f?vrier 2015 23:58:58 UTC+1, Sturla Molden a ?crit?: > On 12/02/15 15:39, Marko Rauhamaa wrote: > > > I write both Py2 and Py3 code, but I keep the two worlds hermetically > > separated from each other. > > > In SciPy world we run the same code on Python 2 and Python 3. > ======== Yes, until... >>> print 'abc?xyz?'.upper() ABC?XYZ? >>> 'abc?xyz'.upper() 'ABC?XYZ' >>> 'abc?def'.upper() 'ABCSSDEF' --- >>> jmUpper('abc?xyz') 'ABC?XYZ' >>> Face it 90% of the Python code is not working out of the box (even "serious applications") making the apps simply not usable. Unfortunately, it's not this nice article, written by a us/ascii user(s), who is (are) not even aware of the situation, that will "save" the situation. Terrible to say, but that's the reality. I was also an "CDC ascii" [*] user. It was 30-35 years ago. Now I'm testing STIX/XITS fonts with a Tex unicode engine (and a little bit more). [*] PASCAL User Manual and Report, Kathleen Jensen, Niklaus Wirth, SPRINGER-VERLAG. jmf From DLipman~nospam~ at Verizon.Net Sat Feb 14 06:50:03 2015 From: DLipman~nospam~ at Verizon.Net (David H. Lipman) Date: Sat, 14 Feb 2015 06:50:03 -0500 Subject: Bad text appearance in IDLE In-Reply-To: <54dc9bee$0$3046$426a34cc@news.free.fr> References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: From: "ast" > Hello > > Here is how text appears in IDLE window > http://www.cjoint.com/data/0BmnEIcxVAx.htm > > Yesterday evening I had not this trouble. It appears > this morning. I restarted my computer with no effect. > > A windows Vista update has been done this morning, with about 10 fixes. I > suspect something gone wrong with this update > > Has somebody an explanation about that ? > > thx http://support.microsoft.com/kb/3013455 KB3013455 has been found to be causing Display Font problems for Windows Vista and Server 2003. A partial mitigation is to use ClearType Font Smoothing. It will greatly help but will not fix all Display Font viewing problems. Right-Click on the Desktop. Choose "Personalize" ---> Window Color and Appearance --> effects Check the box for; "Use the following method to smooth edges of screen fonts" Set it to; ClearType It is recommended to remove this HotFix and then hide it from further download ( at least until it is fixed ). -- Dave Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk http://www.pctipp.ch/downloads/dl/35905.asp From martijn.millecamp at hotmail.com Sat Feb 14 08:19:06 2015 From: martijn.millecamp at hotmail.com (Martijn Millecamp) Date: Sat, 14 Feb 2015 14:19:06 +0100 Subject: pickle error by multiprocessing Message-ID: hey for a schoolproject we had to build a robot and control this robot with a gui to follow a pathWe use multiprocessingand in our group 2 people can run the codebut if i run the code, i got a pickle errorI have a windows 7 and use python 2.7 just like the othersWe asked our prof, but he doesn't know what I can do :sI have already reinstalled python and all the packages Can i do something else? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jessemuzziddu at mail.com Sat Feb 14 10:06:39 2015 From: jessemuzziddu at mail.com (Dieffe Undici) Date: Sat, 14 Feb 2015 07:06:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From jessemuzziddu at mail.com Sat Feb 14 10:07:11 2015 From: jessemuzziddu at mail.com (Dieffe Undici) Date: Sat, 14 Feb 2015 07:07:11 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From priisdk at gmail.com Sat Feb 14 10:08:43 2015 From: priisdk at gmail.com (Poul Riis) Date: Sat, 14 Feb 2015 07:08:43 -0800 (PST) Subject: pyqtgraph window position In-Reply-To: References: <94334ddf-b3ad-4d5a-a11a-237b8b9f494a@googlegroups.com> Message-ID: <1a56deb4-9db8-4474-84e8-1a6e325cfb0a@googlegroups.com> Den l?rdag den 14. februar 2015 kl. 06.59.20 UTC+1 skrev Cousin Stanley: > > I can control the size of my pyqtgraph window below with 'resize'. > > > > But how can I control the position on the screen? > > .... > > Also, try .... > > w.setGeometry( x_pos , y_pos , width , height ) > > > -- > Stanley C. Kitching > Human Being > Phoenix, Arizona Thanks a lot. And how can I control the position of the secondary window in the example below? And how can I make a keyboard interrupt? Poul Riis import pyqtgraph as pg from math import * from time import * import numpy as np win = pg.GraphicsWindow(title="Basic plotting examples") #win.resize(1000,600) win.setGeometry(0,100, 1000,600 ) xx=[] yy=[] datanp = np.random.normal(size=(10,20)) ptr=0 for i in range(0,25): xx.append(i) yy.append(i*i/25.0) xarray=np.array(xx) pw=pg.plot(xx,yy,pen='r') p1 = win.addPlot(title="Extra window") p1.plot(xx,yy,pen='b') j=0 while True: j=j+1 yyy=[] for ii in range(0,len(yy)): yyy.append(sin(pi*(j-ii)/25)*yy[ii]) yarray=np.array(yyy) pw.plot(xarray, yarray, clear=True) pg.QtGui.QApplication.processEvents() From mail at timgolden.me.uk Sat Feb 14 11:17:55 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Sat, 14 Feb 2015 16:17:55 +0000 Subject: pickle error by multiprocessing In-Reply-To: References: Message-ID: <54DF7533.2060309@timgolden.me.uk> On 14/02/2015 13:19, Martijn Millecamp wrote: > for a schoolproject we had to build a robot and control this robot with > a gui to follow a path > We use multiprocessing > and in our group 2 people can run the code > but if i run the code, i got a pickle error > I have a windows 7 and use python 2.7 just like the others > We asked our prof, but he doesn't know what I can do :s > I have already reinstalled python and all the packages > Can i do something else? Without seeing at least some failing code, it's hard to know what to advise. However, multiprocessing does operate differently on Windows than on Linux/OSX. In particular, make sure you've read the Programming Guidelines: https://docs.python.org/2/library/multiprocessing.html#windows TJG From usact2012 at gmail.com Sat Feb 14 14:00:21 2015 From: usact2012 at gmail.com (usact2012 at gmail.com) Date: Sat, 14 Feb 2015 11:00:21 -0800 (PST) Subject: error of drawing map from shape file in Python 3.2 basemap Message-ID: <0e3a2756-0618-4a53-988c-4c8c866426e3@googlegroups.com> I am working on drawing map from shape file in Python 3.2 basemap. But, the longitude values at the bottom axis are only shown partially. Also, all latitude values are missing. Here is my python code. import shapefile as sf import sys import numpy as np import matplotlib.pylab as plt from mpl_toolkits.basemap import Basemap map = Basemap(projection='stere', lon_0=-106.4, lat_0= 31.9, lat_ts = 31.9, \ llcrnrlat=31.7, urcrnrlat= 31.85, \ llcrnrlon=-106.5 , urcrnrlon= -106.1, \ rsphere=6371200., resolution='l', area_thresh=1000) plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k') parallels = np.arange(31.7, 31.85, 0.25) map.drawparallels(parallels, labels=[0, 0, 0, 1] , fontsize=10, labelstyle='+/-', dashes=[2, 2]) meridians = np.arange (-106.5, -106.1, 0.25) map.drawmeridians(meridians, labels=[0, 0, 0, 1], fontsize=10, labelstyle='+/-' , dashes=[2, 2]) No matter how I changed the labels, the latitude/longitude legend values are still missing. From billy.earney at gmail.com Sat Feb 14 19:38:48 2015 From: billy.earney at gmail.com (Billy Earney) Date: Sat, 14 Feb 2015 18:38:48 -0600 Subject: Python implementations in JavaScript In-Reply-To: References: Message-ID: Jonathan, There is Brython. It has been actively developed for a few years now. It is missing some advance python features, but is becoming more python 3.x compliant every day. You should check it out, http://brython.info You ask, are any of the implementations ready to rely on? For Brython, it depends, on what you want to do. It also "hooks" into regular javascript libraries (jquery, etc), if you want to access those. The Brython docs cover much of this. Billy On Fri, Feb 13, 2015 at 5:05 PM, Jonathan Hayward wrote: > What is the relative maturity of different Python implementations in > JavaScript? Are any of the implementations ready to rely on? > > -- > [image: Christos Jonathan Seth Hayward] > Jonathan S. Hayward, a full stack web developer with Python/Django and > AngularJS/jQuery. > > Articles on Technology ? Book: *Django > JavaScript Integration* > > ? *Email * ? Github ? > LinkedIn ? Main Website > ? *Professional Site * > > I invite you to visit my professional site! > > -- > https://mail.python.org/mailman/listinfo/python-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.g.kelly at gmail.com Sat Feb 14 19:39:16 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 14 Feb 2015 17:39:16 -0700 Subject: Floating point "g" format not stripping trailing zeros In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 6:22 PM, Mark Lawrence wrote: > On 14/02/2015 00:11, Ian Kelly wrote: >> >> On Fri, Feb 13, 2015 at 4:05 PM, Mark Lawrence >> wrote: >>> >>> I still think it's a bug as the 'p' being referred to in the OP's >>> original >>> message is "The precision is a decimal number indicating how many digits >>> should be displayed after the decimal point for a floating point value >>> formatted with 'f' and 'F', or before and after the decimal point for a >>> floating point value formatted with 'g' or 'G'". In other words is has >>> nothing to do with the precision of the underlying number. >> >> >> I read that paragraph as describing generally what the precision >> means, with the table below detailing more specifically how it is >> used. I think for 'g' this is just trying to contrast it with 'f' >> where it really is just the number of digits displayed after the >> decimal point. The table entry for 'g' on the other hand quite clearly >> says, "for a given precision p >= 1, this rounds the number to p >> significant digits". >> > > With insignificant trailing zeros removed. So if I'm asking for 15 > significant digits *in the output* I don't expect to see those zeros as I > don't see them as significant. If I did I'd have p set higher. So you think you should have to request 17 significant digits in order to have 15 of them included? From ian.g.kelly at gmail.com Sat Feb 14 20:28:16 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 14 Feb 2015 18:28:16 -0700 Subject: Python implementations in JavaScript In-Reply-To: References: Message-ID: On Fri, Feb 13, 2015 at 4:05 PM, Jonathan Hayward wrote: > > What is the relative maturity of different Python implementations in JavaScript? Are any of the implementations ready to rely on? Brython is about two years old now. Skulpt has been around for at least four years. I don't use either of them so I can't give a comparison. I wouldn't use Pyjamas though at this point as development on it seems to have died since the takeover^H^H^H^H^H^H^H^H fork. From ethan at stoneleaf.us Sat Feb 14 22:37:49 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 14 Feb 2015 19:37:49 -0800 Subject: Python implementations in JavaScript In-Reply-To: References: Message-ID: <54E0148D.7010008@stoneleaf.us> On 02/14/2015 05:28 PM, Ian Kelly wrote: > On Fri, Feb 13, 2015 at 4:05 PM, Jonathan Hayward wrote: >> >> What is the relative maturity of different Python implementations in JavaScript? Are any of the implementations ready to rely on? > > Brython is about two years old now. I remember a post from a few weeks ago that said Brython had worked just fine for their needs. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From dieter at handshake.de Sun Feb 15 01:43:34 2015 From: dieter at handshake.de (dieter) Date: Sun, 15 Feb 2015 07:43:34 +0100 Subject: pickle error by multiprocessing References: Message-ID: <87wq3jektl.fsf@handshake.de> Martijn Millecamp writes: > for a schoolproject we had to build a robot and control this robot with a gui to follow a pathWe use multiprocessingand in our group 2 people can run the codebut if i run the code, i got a pickle errorI have a windows 7 and use python 2.7 just like the othersWe asked our prof, but he doesn't know what I can do :sI have already reinstalled python and all the packages Can i do something else? Usually, multiprocessing should not introduce pickling errors (unless you write the pickles concurrently with other writes or reads). However, you cannot un/pickle everything: the most likely cause for pickling errors is the attempt to pickle something which is not picklable. From dieffedue at gmx.com Sun Feb 15 08:32:20 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. EX SYZ GENEVE.) Date: Sun, 15 Feb 2015 05:32:20 -0800 (PST) Subject: -- redacted -- Message-ID: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> -- redacted -- From dieffedue at gmx.com Sun Feb 15 08:32:52 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. EX SYZ GENEVE.) Date: Sun, 15 Feb 2015 05:32:52 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <8e0e2ad5-6d6c-4d58-a0e4-b5583e1642b5@googlegroups.com> -- redacted -- From ginobusciarello at outlook.com Sun Feb 15 08:54:40 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 15 Feb 2015 05:54:40 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From ginobusciarello at outlook.com Sun Feb 15 08:59:43 2015 From: ginobusciarello at outlook.com (CHE SIA REVOLUCIOOON!!!) Date: Sun, 15 Feb 2015 05:59:43 -0800 (PST) Subject: -- redacted -- In-Reply-To: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: -- redacted -- From blaxxton at yahoo.com Sun Feb 15 10:40:41 2015 From: blaxxton at yahoo.com (Blaxton) Date: Sun, 15 Feb 2015 15:40:41 +0000 (UTC) Subject: Creating Python RPM on AIX Message-ID: <1876218830.6043349.1424014841189.JavaMail.yahoo@mail.yahoo.com> Hi Has any one created Python RPM on AIX ? I am using the spec file that comes with Python source file and here is the problems I was facing: README file says download the bz2 version, but only gziped and xz version available from Python website so had to convert the .tgz file to bz2 and copy it over to SOURCES directory. %define config_binsuffix? was set to 2.6 and not sure why it should be 2.6 when I working on 3.4 version. had to change it to to pass the error. %define config_binsuffix none Misc/cheatsheet seems to be decommissioned in new version of Python but still in spec file so commented it. The last error that I am receiving and can't figure the resolution is below: Processing files: python-devel-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4/*.h File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/lib/python3.4/config PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2 Processing files: python-tools-3.4.2-AIX71 File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/pydoc File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/smtpd.py PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2-AIX71 running rpm -bl option to see missing files are below: rpm -bl /opt/freeware/src/packages/SPECS/python-3.4.spec Processing files: python-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/opt/freeware/man/man1/python.1* File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4 Processing files: python-devel-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4/*.h File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/lib/python3.4/config PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2 Processing files: python-tools-3.4.2-AIX71 File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/pydoc File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/smtpd.py PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2-AIX71 Any idea as how I can fix this ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Sun Feb 15 11:46:04 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sun, 15 Feb 2015 08:46:04 -0800 Subject: pickle error by multiprocessing Message-ID: <1424018764.66139.BPMail_high_carrier@web163801.mail.gq1.yahoo.com> ---------------------------- On Sun, Feb 15, 2015 7:43 AM CET dieter wrote: >Martijn Millecamp writes: > >> for a schoolproject we had to build a robot and control this robot with a gui to follow a pathWe use multiprocessingand in our group 2 people can run the codebut if i run the code, i got a pickle errorI have a windows 7 and use python 2.7 just like the othersWe asked our prof, but he doesn't know what I can do :sI have already reinstalled python and all the packages Can i do something else? > >Usually, multiprocessing should not introduce pickling errors (unless >you write the pickles concurrently with other writes or reads). > >However, you cannot un/pickle everything: the most likely cause >for pickling errors is the attempt to pickle something which is not picklable. Does it also occurr when you use the exact same Python version? Long shot, but hav you seen this: http://bugs.python.org/issue21204 >-- >https://mail.python.org/mailman/listinfo/python-list From skip.montanaro at gmail.com Sun Feb 15 13:13:55 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 15 Feb 2015 12:13:55 -0600 Subject: Go Ahead, Make my PEP... (was: PEP 441 - Improving Python ZIP Application Support) Message-ID: >From over yonder, on python-dev... > ---------- Forwarded message ---------- > From: Steve Dower > Subject: Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support > > "Go ahead, make my pep." > > We should make a python-dev t-shirt with this on it :) +1 from me as QOTW. Skip From marfig at gmail.com Sun Feb 15 15:04:51 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Sun, 15 Feb 2015 21:04:51 +0100 Subject: Flagging classes as not intended for direct initialization Message-ID: Hello everyone, [Python 3.X] I have the following factory model for the initialization of a class tree (code abbreviated for simplicity). # item.py class BadItemType(Exception): pass class Item: def __init__(self, _data): class Container(Item): def __init__(self, _data): Item.__init__(self, _data) # ... class Tool(Item): def __init__(self, _data): Item.__init__(self, _data) # ... def spawn(type_, id_): if type_ not in Item.__subclasses__(): raise BadItemType() # ... return type_(data) I'd like to know your opinions on an acceptable way to flag the Item class and its derived classes so users of this module know they should avoid instantiating them directly. Other than the obvious notes on the classes docstrings, is it good enough I mark the constructor argument as an implementation variable as I did? From kai.peters at gmail.com Sun Feb 15 15:17:29 2015 From: kai.peters at gmail.com (KP) Date: Sun, 15 Feb 2015 12:17:29 -0800 (PST) Subject: How to install PIL or PILLOW on OS X Yosemite? Message-ID: Hi, just upgraded my Mac Mini to Yosemite and have never dabbled in Python on this OS. I see it has Python 2.7.6 installed. When I do something like from PIL import ImageFont, ImageDraw it tells me that it cannot find PIL How do I install this on Yosemite? Any pointers much appreciated KP From pecore at pascolo.net Sun Feb 15 16:09:01 2015 From: pecore at pascolo.net (giacomo boffi) Date: Sun, 15 Feb 2015 22:09:01 +0100 Subject: Python discussed in Nature References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87pp9a287m.fsf@pascolo.net> Chris Angelico writes: > On Sat, Feb 14, 2015 at 6:40 AM, John Ladasky > wrote: >> The default font that the Geany program editor uses on my Ubuntu >> system renders everything I've tried. When I look up that font in >> Geany's Preferences menu, it is called, simply, "monospace". >> > > That's a font alias. Unfortunately, I've never yet figured out a > straight-forward way to snap the pointer; when you know it, it's easy... % fc-match mono DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book" % From rosuav at gmail.com Sun Feb 15 17:16:30 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 16 Feb 2015 09:16:30 +1100 Subject: Python discussed in Nature In-Reply-To: <87pp9a287m.fsf@pascolo.net> References: <54dc6870$0$12981$c3e8da3$5496439d@news.astraweb.com> <6322f4fa-c8af-4051-b6c6-ff953bd928ca@googlegroups.com> <3bfcd265-fb89-431c-b9b6-94b24b2ef025@googlegroups.com> <54dd6b76$0$12998$c3e8da3$5496439d@news.astraweb.com> <87pp9a287m.fsf@pascolo.net> Message-ID: On Mon, Feb 16, 2015 at 8:09 AM, giacomo boffi wrote: >> That's a font alias. Unfortunately, I've never yet figured out a >> straight-forward way to snap the pointer; > > when you know it, it's easy... > > % fc-match mono > DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book" > % Learn something new every day! Thanks Giacomo! ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 15 20:50:53 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 16 Feb 2015 12:50:53 +1100 Subject: Flagging classes as not intended for direct initialization References: Message-ID: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Mario Figueiredo wrote: > Hello everyone, > > [Python 3.X] > > I have the following factory model for the initialization of a class > tree (code abbreviated for simplicity). > > # item.py > class BadItemType(Exception): > pass If this is a type error, why aren't you using TypeError? Or at least inheriting from TypeError? class BadItemType(TypeError): pass > class Item: > def __init__(self, _data): > > class Container(Item): > def __init__(self, _data): > Item.__init__(self, _data) > # ... > > class Tool(Item): > def __init__(self, _data): > Item.__init__(self, _data) > # ... Unless you have good reason not to, you should use super rather than directly call the superclass. https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ > def spawn(type_, id_): > if type_ not in Item.__subclasses__(): > raise BadItemType() > # ... > return type_(data) > > I'd like to know your opinions on an acceptable way to flag the Item > class and its derived classes so users of this module know they should > avoid instantiating them directly. > > Other than the obvious notes on the classes docstrings, is it good > enough I mark the constructor argument as an implementation variable as > I did? Oh, so you did. No, that's not good enough. I never even noticed that, and if I had noticed it, I would have guessed that you had some bizarre naming scheme and didn't understand the Python underscore conventions. I never would have guessed that you meant "don't instantiate this class". Are users supposed to subclass Item themselves? If so, you should make Item an ABC (Abstract Base Class): class ItemABC: def __init__(self, data): if type(self) is ItemABC: raise TypeError("You must subclass ItemABC") But if users aren't supposed to use the classes *at all*, but only use the spawn factory function, you don't have a lot of good choices in pure Python code. Python isn't designed to be that restrictive. The *simplest thing which will work* is simply not worry about it. You should consider the factory function to be merely for convenience. It might be *really* convenient, but if the user wants to ignore the factory and reinvent the wheel, why should you care? Just document the fact that people are expected to use the factory, but don't expressly prohibit the use of the classes directly. Otherwise, you can flag the classes with single underscores to mark them as private implementation details: class _Item: ... then declare "consenting adults". If your users decide to instantiate the classes directly, you can't really stop them, so if they blow their foot off they have nobody to blame but themselves. If you want to be a little more strict, you can use a little obfuscation to discourage direct use. Note that in this case you *must* inherit from object in Python 2. In Python 3 it doesn't matter. # Untested class Item(object): def __init__(self): raise TypeError('do not instantiate') @classmethod def _my_secret_constructor(cls, data): obj = super(Item, cls).__new__(cls) obj.data = data return obj def spawn(type_, id_): # error checking as before return type_._my_secret_constructor(data) but of course even here if your users *really* want to instantiate the class, they can do so. -- Steven From nad at acm.org Sun Feb 15 23:17:20 2015 From: nad at acm.org (Ned Deily) Date: Sun, 15 Feb 2015 20:17:20 -0800 Subject: How to install PIL or PILLOW on OS X Yosemite? References: Message-ID: In article , KP wrote: > just upgraded my Mac Mini to Yosemite and have never dabbled in Python on > this OS. > > I see it has Python 2.7.6 installed. > > When I do something like > > from PIL import ImageFont, ImageDraw > > it tells me that it cannot find PIL > > How do I install this on Yosemite? Suggestions: stick with Pillow which is the current, maintained fork of the venerable PIL. Decide whether you want to use Python 3 or Python 2. PIL/Pillow installation on OS X is more involved than on some other platforms because it depends on a number of third-party C libraries that are not shipped by Apple in OS X so you need to find another source for them. Rather than trying to build and install everything yourself or downloading a Pillow or PIL installer, I suggest picking one of the several fine distributors of open source packages for OS X and installing everything you need from them (including an up-to-date Python 2 or 3) and for your future needs beyond Pillow; options include Homebrew, MacPorts, Anaconda, Fink, and others. Once you've installed the base framework for the package manager you choose, installing something like Pillow and all of its dependencies is often just a one-line command. It may take a little while to get used to the quirks of the package manager you choose but, if you are going to use OS X for development with Python or many other languages, that time spent will be repaid many times over. -- Ned Deily, nad at acm.org From __peter__ at web.de Mon Feb 16 02:42:43 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 16 Feb 2015 08:42:43 +0100 Subject: Flagging classes as not intended for direct initialization References: Message-ID: Mario Figueiredo wrote: > Hello everyone, > > [Python 3.X] > > I have the following factory model for the initialization of a class > tree (code abbreviated for simplicity). > > # item.py > class BadItemType(Exception): > pass > > class Item: > def __init__(self, _data): > > class Container(Item): > def __init__(self, _data): > Item.__init__(self, _data) > # ... > > class Tool(Item): > def __init__(self, _data): > Item.__init__(self, _data) > # ... > > def spawn(type_, id_): > if type_ not in Item.__subclasses__(): > raise BadItemType() > # ... > return type_(data) > > I'd like to know your opinions on an acceptable way to flag the Item > class and its derived classes so users of this module know they should > avoid instantiating them directly. > > Other than the obvious notes on the classes docstrings, is it good > enough I mark the constructor argument as an implementation variable as > I did? You can delete the class from the namespace: class Item: ... def spawn_item(id, class_=Item): ... return class_(data) del Item From marfig at gmail.com Mon Feb 16 05:55:45 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 16 Feb 2015 11:55:45 +0100 Subject: Flagging classes as not intended for direct initialization References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: In article <54e14cfe$0$12997$c3e8da3$5496439d at news.astraweb.com>, steve+comp.lang.python at pearwood.info says... > > If this is a type error, why aren't you using TypeError? Or at least > inheriting from TypeError? > Yeah. It's not that I'm gaining much from that abstraction. It's not even an abstraction, it's just masking since there really won't be any specific logic to that exception. You are right. Removed it. > > Unless you have good reason not to, you should use super rather than > directly call the superclass. > > https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ It's not been an easy ride trying to decide whether or not to use super. I started learning python from a Mark Lutz book that advised me against it. And on the web I get all sorts of information pro and against. Seems like everyone has an opinion and theirs the correct one. I have not delved deep into that construct to decide for myself. But thanks for the link. It actually addresses more than most and builds a pretty good case for it. > > Are users supposed to subclass Item themselves? If so, you should make Item > an ABC (Abstract Base Class): > > class ItemABC: > def __init__(self, data): > if type(self) is ItemABC: > raise TypeError("You must subclass ItemABC") > Ooh! Was forgetting about that. Thanks. > > But if users aren't supposed to use the classes *at all*, but only use the > spawn factory function, you don't have a lot of good choices in pure Python > code. Python isn't designed to be that restrictive. > > The *simplest thing which will work* is simply not worry about it. You > should consider the factory function to be merely for convenience. It might > be *really* convenient, but if the user wants to ignore the factory and > reinvent the wheel, why should you care? Just document the fact that people > are expected to use the factory, but don't expressly prohibit the use of > the classes directly. Yeah. I think this is best. It is one of the things I like about Python. This mentality not only simplifies code but also gives room for tinkerers to do their thing without unnecessary barriers. I was more concerned with visibility and letting users know how to properly use that class tree. And docstrings are good enough. The data argument used in the classes initialization is a dict containing the necessary initialization data. So without reading the class source and documentation, a user has no way of knowing how to initialize those classes directly, anyways. Thanks. From robert.kern at gmail.com Mon Feb 16 06:06:19 2015 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 16 Feb 2015 11:06:19 +0000 Subject: line_profiler: what am I doing wrong? In-Reply-To: References: Message-ID: On 2015-02-13 13:35, Neal Becker wrote: > Robert Kern wrote: > >> @profile >> def run(): >> pass >> >> run() > > No, this doesn't work either. Same failure > > kernprof -l test_prof.py > Wrote profile results to test_prof.py.lprof > Traceback (most recent call last): > File "/home/nbecker/.local/bin/kernprof", line 9, in > load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() > File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line 221, > in main > execfile(script_file, ns, ns) > File "test_prof.py", line 1, in > @profile > NameError: name 'profile' is not defined Ah, do you have the package `future` installed? https://github.com/rkern/line_profiler/issues/12 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ian.g.kelly at gmail.com Mon Feb 16 06:59:48 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 16 Feb 2015 04:59:48 -0700 Subject: Flagging classes as not intended for direct initialization In-Reply-To: References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 16, 2015 at 3:55 AM, Mario Figueiredo wrote: > In article <54e14cfe$0$12997$c3e8da3$5496439d at news.astraweb.com>, > steve+comp.lang.python at pearwood.info says... >> Unless you have good reason not to, you should use super rather than >> directly call the superclass. >> >> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ > > It's not been an easy ride trying to decide whether or not to use super. > I started learning python from a Mark Lutz book that advised me against > it. And on the web I get all sorts of information pro and against. Seems > like everyone has an opinion and theirs the correct one. > > I have not delved deep into that construct to decide for myself. But > thanks for the link. It actually addresses more than most and builds a > pretty good case for it. There's nothing inherently bad about super itself, other than the name being slightly misleading. For single inheritance hierarchies, there is no reason at all not to use it -- particularly in Python 3 where you don't even have to pass it arguments. It was designed to resolve the diamond problem in multiple inheritance situations, though, and while super does a fine job of that, it doesn't solve the basic problem that multiple inheritance is usually a bad idea to start with that ends up as a complicated, confusing mess. Use super for superclass calls, avoid multiple inheritance, and you'll have no problems. From julien.levasseur at gmail.com Mon Feb 16 07:17:49 2015 From: julien.levasseur at gmail.com (julien levasseur) Date: Mon, 16 Feb 2015 13:17:49 +0100 Subject: PythonMagick on Windows DLL load failed Message-ID: I am using Python 2.7 on Windows 8.1. Python 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32 I installed ImageMagick from imagemagick.org Then installed PythonMagick with pip from http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonmagick When I import PythonMagick it says: Traceback (most recent call last): File "magic.py", line 2, in import PythonMagick File "C:\Python27\lib\site-packages\PythonMagick\__init__.py", line 1, in from . import _PythonMagick ImportError: DLL load failed: The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail. Any clues on where do look? Question on stackoverflow http://stackoverflow.com/questions/28538973/pythonmagick-on-windows-dll-load-failed -- Julien Levasseur -------------- next part -------------- An HTML attachment was scrubbed... URL: From roy at panix.com Mon Feb 16 08:14:00 2015 From: roy at panix.com (Roy Smith) Date: Mon, 16 Feb 2015 08:14:00 -0500 Subject: Flagging classes as not intended for direct initialization References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: In article , Mario Figueiredo wrote: > It's not been an easy ride trying to decide whether or not to use super. > I started learning python from a Mark Lutz book that advised me against > it. I'm curious, what were the arguments against it? From marfig at gmail.com Mon Feb 16 08:47:53 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 16 Feb 2015 14:47:53 +0100 Subject: Flagging classes as not intended for direct initialization References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, 16 Feb 2015 08:14:00 -0500, Roy Smith wrote: >In article , > Mario Figueiredo wrote: > >> It's not been an easy ride trying to decide whether or not to use super. >> I started learning python from a Mark Lutz book that advised me against >> it. > >I'm curious, what were the arguments against it? The following is quoted from Learning Python, 5th Edition: >Java programmers may especially be interested to know that Python also has a super >built-in function that allows calling back to a superclass?s methods more generically? >but it?s cumbersome to use in 2.X; differs in form between 2.X and 3.X; relies on unusual >semantics in 3.X; works unevenly with Python?s operator overloading; and does not >always mesh well with traditionally coded multiple inheritance, where a single superclass call won?t suffice. > >In its defense, the supercall has a valid use case too?cooperative same-named method >dispatch in multiple inheritance trees?but it relies on the ?MRO? ordering of classes, >which many find esoteric and artificial; unrealistically assumes universal deployment >to be used reliably; does not fully support method replacement and varying argument >lists; and to many observers seems an obscure solution to a use case that is rare in real >Python code. > >Because of these downsides, this book prefers to call superclasses by explicit name >instead of super, recommends the same policy for newcomers, and defers presenting >supe until Chapter 32. > [...] >And to any Java programmers in the audience: I suggest resisting the temptation to use >Python?s superuntil you?ve had a chance to study its subtle implications. Once you >step up to multiple inheritance, it?s not what you think it is, and more than you probably >expect. The class it invokes may not be the superclass at all, and can even vary per >context. Or to paraphrase a movie line: Python?s superis like a box of chocolates?you >never know what you?re going to get! Later, in chapter 32 where super is finally introduced, you can read the following: >As noted in Chapter 28 and Chapter 29, Python has a superbuilt-in function that can >be used to invoke superclass methods generically, but was deferred until this point of >the book. This was deliberate?because superhas substantial downsides in typical >code, and a sole use case that seems obscure and complex to many observers, most >beginners are better served by the traditional explicit-name call scheme used so far. See >the sidebar ?What About super?? on page 831in Chapter 28for a brief summary of >the rationale for this policy. >The Python community itself seems split on this subject, with online articles about it >running the gamut from ?Python?s Super Considered Harmful? to ?Python?s super() >considered super!? >Frankly, in my live classes this call seems to be most often of interest >to Java programmers starting to use Python anew, because of its conceptual similarity >to a tool in that language (many a new Python feature ultimately owes its existence to >programmers of other languages bringing their old habits to a new model). Python?s >superis not Java?s?it translates differently to Python?s multiple inheritance, and has >a use case beyond Java?s?but it has managed to generate both controversy and >misunderstanding since its conception. > >In its defense, this call does have a valid use case too?cooperative same-named method >dispatch in diamond multiple inheritance trees?but it seems to ask a lot of newcomers. >It requires that superbe used universally and consistently (if not neurotically), much >like __slots__discussed earlier; relies on the arguably obscure MRO algorithm to order >calls; and addresses a use case that seems far more the exception than the norm in >Python programs. In this role, superseems an advanced tool based upon esoteric principles, >which may be beyond much of Python?s audience, and seems artificial to real >program goals. That aside, its expectation of universal use seems unrealistic for the vast >amount of existing Python code. From David.Aldrich at EMEA.NEC.COM Mon Feb 16 09:03:55 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Mon, 16 Feb 2015 14:03:55 +0000 Subject: A question about a list and subprocess.check_call() Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B93AD15@EX10MBX02.EU.NEC.COM> Hi I wonder if someone could help me with this problem please. I am writing a Python script that builds and tests a C++ program on Linux. The build options depend on the test, so I have encapsulated the 'make' call in a Python function: def build(build_options=''): if len(build_options): subprocess.check_call(['make',build_options]) else: subprocess.check_call('make') This works fine if I call: build() or build('flagA=true') The latter gives: make flagA=true which is correct. However, I now want to call make with two flags: make flagA=true flagB=true I tried calling: build('flagA=true flagB=true') which did indeed result in: make flagA=true flagB=true but 'make' ignored the second option. So I think that the list that was passed to subprocess.check_call() was incorrect. In summary, I want to pass a list to build(), which by default should be empty, and pass that list on to subprocess.check_call() with 'make' as the first element of the list. Any ideas please? Best regards David -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Mon Feb 16 09:11:01 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Feb 2015 01:11:01 +1100 Subject: Flagging classes as not intended for direct initialization In-Reply-To: References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 17, 2015 at 12:47 AM, Mario Figueiredo wrote: > The following is quoted from Learning Python, 5th Edition: > >>Java programmers may especially be interested to know that Python also has a super >>built-in function that allows calling back to a superclass?s methods more generically? >>but it?s cumbersome to use in 2.X; differs in form between 2.X and 3.X; relies on unusual >>semantics in 3.X; works unevenly with Python?s operator overloading; and does not >>always mesh well with traditionally coded multiple inheritance, where a single superclass call won?t suffice. >> >>In its defense, the supercall has a valid use case too?cooperative same-named method >>dispatch in multiple inheritance trees?but it relies on the ?MRO? ordering of classes, >>which many find esoteric and artificial; unrealistically assumes universal deployment >>to be used reliably; does not fully support method replacement and varying argument >>lists; and to many observers seems an obscure solution to a use case that is rare in real >>Python code. >> Let's take a step back here. super() depends on the "MRO" ordering of classes. Yes, but that's not super()'s decision - that is, quite fundamentally, how Python classes work. Here's a quick look at the MRO; this should work equally in Py2 and Py3: class Top(object): def foo(self): print("I am top - foo") def top(self): print("I am top - top") class Left(Top): def foo(self): print("I am left - foo") super(Left, self).foo() def leftright(self): print("I am left - leftright") class Right(Top): def foo(self): print("I am right - foo") super(Right, self).foo() def leftright(self): print("I am right - leftright") class Diamond1(Left, Right): def foo(self): print("I am diamond1 - foo") super(Diamond1, self).foo() # No leftright method class Diamond2(Left, Right): def foo(self): print("I am diamond2 - foo") super(Diamond2, self).foo() def leftright(self): print("I am diamond2 - leftright") super(Diamond2, self).leftright() print(Diamond1.__mro__) print(Diamond2.__mro__) d1 = Diamond1() d2 = Diamond2() d1.foo() d2.foo() d1.leftright() d2.leftright() Note that calling d1.leftright() calls the method from Left, not the one from Right; and calling super() from inside Diamond2's leftright does the exact same thing. So what you have here is not "super() is weird", but "multiple inheritance is messy, and this is how Python handles it". I agree with the "cumbersome to use in 2.x", but it's not actually different in syntax across the two - correct me if I'm wrong, but I believe that if you write it out in full, your code is guaranteed to work equally in both branches. So you can't complain about the difference AND complain about the old syntax being clunky; the difference is precisely the solution to the clunkiness. "unrealistically assumes universal deployment to be used reliably". Hmm. We're talking here about a solution to diamond inheritance. Can you name *any* solution to that problem which does not require some careful management by all involved classes? In any language at all? Multiple inheritance is the issue. If you want to defer anything till a later chapter, defer that. Once you decide to learn about MI, you just have to knuckle down and learn Python's particular flavour of MI, because it's different from C++'s (both default and virtual inheritance), different from Java's ("there is no MI"), different from Pike's, etc. And when you learn Python's MI, what you'll be learning is the MRO; super() just comes along for the ride. Explicit naming of superclasses is fragile for several reasons. Firstly, it breaks the parallel between "method which calls its superclass method" and "absence of method, and implicit referral to superclass" (as the latter guarantees to use the MRO); and secondly, you end up writing your superclass name a lot of times. Though, to be honest, the Py2 super() syntax has the same issue, just with your own class name. With the Py3 shorthand syntax, you're not rewriting everything all over the place. ChrisA From marfig at gmail.com Mon Feb 16 10:00:08 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 16 Feb 2015 16:00:08 +0100 Subject: Flagging classes as not intended for direct initialization References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> Message-ID: <4cv3eat2d1rfr049alntig976btm3te1sb@4ax.com> On Tue, 17 Feb 2015 01:11:01 +1100, Chris Angelico wrote: >So what you have here is not "super() is weird", but "multiple >inheritance is messy, and this is how Python handles it". > I'd say the diamond pattern is messy. MI is otherwise a pretty peaceful kid without it. I don't find the C3 linearization method particularly hard to understand either. Contrary to Mark, I don't look at MRO as some esoteric property of Python. Heck, took me more time to understand and take advantage of tuple packing and unpacking, than it took me to understand MRO. What may make MRO complex is large hierarchic trees, defined across multiple modules. But that really is complex under any method resolution model we can think of. Under those circumstances, proper documentation is the solution whether one is using C3 linearization (which really isn't exclusive to Python) or something else, like explicit naming or depth search rules. What really put me off was instead the mention to lack of support for variadic methods and some apparent problems with operator overloading. I reasoned that for the sake of consistency, I'm probably better using super only when I need its specific set of features. >Explicit naming of superclasses is fragile for several reasons. >Firstly, it breaks the parallel between "method which calls its >superclass method" and "absence of method, and implicit referral to >superclass" (as the latter guarantees to use the MRO); I agree. Super is more explicit than an explicit method call. Which is ironic. Both in fact can coexist, with explicit method calls being used to denote an MRO deviation, for instance. But frankly, I don't see the breaking of that parallel as a problem. For the most part, you can code in complete ignorance of MRO, even in the presence of MRI. MI name clashing is something that we ought to avoid anyways and it is so rare we ever have to confront it, that making a case for super on that basis is a bit weak. > and secondly, >you end up writing your superclass name a lot of times. Though, to be >honest, the Py2 super() syntax has the same issue, just with your own >class name. With the Py3 shorthand syntax, you're not rewriting >everything all over the place. I do like the super syntax better. I rewrote my code to include it, as advised earlier. After looking at the link Steven provided and reading a bit more about here, I decided to go with it instead in in single hierarchy classes. But I have to agree with Mark that the case for super is a bit weak. The arguments for it are based on MI edge cases which really doesn't translate well to "super is better than explicit method calling". From __peter__ at web.de Mon Feb 16 10:19:56 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 16 Feb 2015 16:19:56 +0100 Subject: A question about a list and subprocess.check_call() References: <41302A7145AC054FA7A96CFD03835A0A0B93AD15@EX10MBX02.EU.NEC.COM> Message-ID: David Aldrich wrote: > Hi > > I wonder if someone could help me with this problem please. I am writing > a Python script that builds and tests a C++ program on Linux. The build > options depend on the test, so I have encapsulated the 'make' call in a > Python function: > > def build(build_options=''): > if len(build_options): > subprocess.check_call(['make',build_options]) > else: > subprocess.check_call('make') > > This works fine if I call: > > build() > or > build('flagA=true') > > The latter gives: > > make flagA=true > > which is correct. > > However, I now want to call make with two flags: > > make flagA=true flagB=true > > I tried calling: > > build('flagA=true flagB=true') > > which did indeed result in: > > make flagA=true flagB=true > > but 'make' ignored the second option. So I think that the list that was > passed to subprocess.check_call() was incorrect. That's because you effectively call subprocess.check_call(["make", "flagA=true flabB=true"]) i. e. you are providing a single parameter "flagA=true flabB=true" instead of two separate "flagA=true" and "flabB=true" > In summary, I want to pass a list to build(), which by default should be > empty, and pass that list on to subprocess.check_call() with 'make' as the > first element of the list. > > Any ideas please? The straightforward approach is to pass a list or tuple: def build(build_options=()): subprocess_check_call(("make",) + build_options) build(("flagA=true", "flagB=true")) Allowing separate arguments may be more convenient to use: def build(*build_options): subprocess.check_call(("make",) + build_options) build("flagA=true", "flagB=true") Or, if the order of flags doesn't matter: def build(*args, **kw): subprocess.check_call(("make",) + args + tuple("{}={}".format(*p) for p in kw.items())) build(flagA="true", flagB="true") From rosuav at gmail.com Mon Feb 16 10:29:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Feb 2015 02:29:44 +1100 Subject: Flagging classes as not intended for direct initialization In-Reply-To: <4cv3eat2d1rfr049alntig976btm3te1sb@4ax.com> References: <54e14cfe$0$12997$c3e8da3$5496439d@news.astraweb.com> <4cv3eat2d1rfr049alntig976btm3te1sb@4ax.com> Message-ID: On Tue, Feb 17, 2015 at 2:00 AM, Mario Figueiredo wrote: > On Tue, 17 Feb 2015 01:11:01 +1100, Chris Angelico > wrote: > >>So what you have here is not "super() is weird", but "multiple >>inheritance is messy, and this is how Python handles it". >> > > I'd say the diamond pattern is messy. MI is otherwise a pretty > peaceful kid without it. In Python, everything ultimately inherits from 'object', so any MI will become a diamond. You absolutely (and correctly) expect every object, regardless of its inheritance tree, to respond to methods like __repr__; if you multiply inherit, somewhere along the way, you're going to get a __repr__ function. (Actually, with that particular example, it's probably important to override, rather than depend on the superclasses.) > I don't find the C3 linearization method particularly hard to > understand either. Contrary to Mark, I don't look at MRO as some > esoteric property of Python. Heck, took me more time to understand and > take advantage of tuple packing and unpacking, than it took me to > understand MRO. I had to get my head around the way the calls can move "across" the hierarchy, rather than simply up it. It means that "calling the superclass" might actually call a class that's completely unrelated to the one you're doing this from. In C++, calling the superclass can only ever call a parent of the current class. (Pike has a completely different model, though. Instead of calling one parent method, you can call all of them! It's very cool... as long as you know what you're doing when you make a diamond.) > What may make MRO complex is large hierarchic trees, defined across > multiple modules. But that really is complex under any method > resolution model we can think of. Precisely. It's MI that makes the complexity possible, and the MRO is one of a number of ways to tame that. > What really put me off was instead the mention to lack of support for > variadic methods and some apparent problems with operator overloading. > I reasoned that for the sake of consistency, I'm probably better using > super only when I need its specific set of features. I've no idea what he means by that, unless he's talking about methods with the same name and different argument signatures. If you have a structure like that, then any MI system is going to have problems. Possibly the best way to cope is to use **args everywhere, pop out the ones you care about, and pass the rest on. >>Explicit naming of superclasses is fragile for several reasons. >>Firstly, it breaks the parallel between "method which calls its >>superclass method" and "absence of method, and implicit referral to >>superclass" (as the latter guarantees to use the MRO); > > I agree. Super is more explicit than an explicit method call. Which is > ironic. Both in fact can coexist, with explicit method calls being > used to denote an MRO deviation, for instance. > > But frankly, I don't see the breaking of that parallel as a problem. > For the most part, you can code in complete ignorance of MRO, even in > the presence of MRI. MI name clashing is something that we ought to > avoid anyways and it is so rare we ever have to confront it, that > making a case for super on that basis is a bit weak. Sure, but I would say that the normal, default assumption should be that it's easy to do nothing. I should be able to adorn a parent method as easily as I can adorn with a function decorator. Compare: def announce(func): @wraps(func): def inner(*a,**kw): print("Calling %s!"%func.__name__) return func(*a,**kw) return inner There's a bit of boilerplate to make a pass-through decoration like that, but it's pretty straight-forward. Here's the equivalent Py3 class: class blah(...): def method(self, *a, **kw): print("Calling method on %r!"%self) return super().method(*a, **kw) If you use this model with single inheritance, you could simply name the parent class in the last line, and it'd be fine. But then you subclass this, and suddenly your adornment results in a completely different method being called - that's going to be seriously surprising. And you don't have to change anything in this class itself for the wrapper to suddenly break. ChrisA From David.Aldrich at EMEA.NEC.COM Mon Feb 16 12:07:28 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Mon, 16 Feb 2015 17:07:28 +0000 Subject: A question about a list and subprocess.check_call() In-Reply-To: References: <41302A7145AC054FA7A96CFD03835A0A0B93AD15@EX10MBX02.EU.NEC.COM> Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B941E92@EX10MBX02.EU.NEC.COM> Hi Peter Thanks very much for your reply. I have added one more question below. > The straightforward approach is to pass a list or tuple: > > def build(build_options=()): > subprocess_check_call(("make",) + build_options) > > build(("flagA=true", "flagB=true")) This looks fine - I am trying it. I would like to display on the console the entire make command, so I have done this: def build(build_options=()): make_command = 'make '.join(map(build_options)) print('Build command: ' + make_command) subprocess.check_call(("make",)+build_options) but I get error: make_command = 'make '.join(map(build_options)) TypeError: map() must have at least two arguments. What would be the correct way to concatenate and display the elements in the tuple please? Best regards David From python at mrabarnett.plus.com Mon Feb 16 13:29:54 2015 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 16 Feb 2015 18:29:54 +0000 Subject: A question about a list and subprocess.check_call() In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B941E92@EX10MBX02.EU.NEC.COM> References: <41302A7145AC054FA7A96CFD03835A0A0B93AD15@EX10MBX02.EU.NEC.COM> <41302A7145AC054FA7A96CFD03835A0A0B941E92@EX10MBX02.EU.NEC.COM> Message-ID: <54E23722.6030301@mrabarnett.plus.com> On 2015-02-16 17:07, David Aldrich wrote: > Hi Peter > > Thanks very much for your reply. I have added one more question below. > >> The straightforward approach is to pass a list or tuple: >> >> def build(build_options=()): >> subprocess_check_call(("make",) + build_options) >> >> build(("flagA=true", "flagB=true")) > > This looks fine - I am trying it. > > I would like to display on the console the entire make command, so I have done this: > > def build(build_options=()): > make_command = 'make '.join(map(build_options)) > print('Build command: ' + make_command) > subprocess.check_call(("make",)+build_options) > > but I get error: > > make_command = 'make '.join(map(build_options)) > TypeError: map() must have at least two arguments. > > What would be the correct way to concatenate and display the elements in the tuple please? > make_command = 'make ' + ' '.join(build_options) You don't need 'map' because you're not doing anything to the options before concatenating them. The string before the .join is used as a separator between the joined strings. From tvandame at financegenius.com Mon Feb 16 13:34:41 2015 From: tvandame at financegenius.com (Travis VanDame) Date: Mon, 16 Feb 2015 10:34:41 -0800 (PST) Subject: Python & Peewee Query Example Needed Message-ID: <784409a1-84f6-42d1-b35e-c1f6306060d6@googlegroups.com> I'm new to python and peewee and was looking for an example on how to query a mysql table with a datetime column only returning rows that are 30 days old. From __peter__ at web.de Mon Feb 16 13:39:52 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 16 Feb 2015 19:39:52 +0100 Subject: A question about a list and subprocess.check_call() References: <41302A7145AC054FA7A96CFD03835A0A0B93AD15@EX10MBX02.EU.NEC.COM> <41302A7145AC054FA7A96CFD03835A0A0B941E92@EX10MBX02.EU.NEC.COM> Message-ID: David Aldrich wrote: > Hi Peter > > Thanks very much for your reply. I have added one more question below. > >> The straightforward approach is to pass a list or tuple: >> >> def build(build_options=()): >> subprocess_check_call(("make",) + build_options) >> >> build(("flagA=true", "flagB=true")) > > This looks fine - I am trying it. > > I would like to display on the console the entire make command, so I have > done this: > > def build(build_options=()): > make_command = 'make '.join(map(build_options)) > print('Build command: ' + make_command) > subprocess.check_call(("make",)+build_options) > > but I get error: > > make_command = 'make '.join(map(build_options)) > TypeError: map() must have at least two arguments. > > What would be the correct way to concatenate and display the elements in > the tuple please? Hm, what do you expect map() to achieve? And "make " is not really the string you want as a separator... You can join the strings in build_options with >>> build_options = "foo", "bar", "baz" >>> " ".join(build_options) 'foo bar baz' >>> "make " + " ".join(build_options) 'make foo bar baz' But you shouldn't do this as you have already learnt that spaces do not separate arguments; in fact by default check_call() doesn't even use the shell. An example to drive the point home: >>> from subprocess import check_call >>> check_call(["python3", "-c", "import sys; print(sys.argv[1:])", "foo", "bar"]) ['foo', 'bar'] 0 >>> check_call(["python3", "-c", "import sys; print(sys.argv[1:])", "foo bar"]) ['foo bar'] Both invocations would appear to issue the same command in your debugging print() call. list2cmdline() does a bit better; it resembles the command issued if the shell were used: >>> print(subprocess.list2cmdline(["make"] + build_options)) make foo "bar baz" From tvandame at financegenius.com Mon Feb 16 14:08:18 2015 From: tvandame at financegenius.com (Travis VanDame) Date: Mon, 16 Feb 2015 11:08:18 -0800 (PST) Subject: Python & Peewee Query Example Needed In-Reply-To: <784409a1-84f6-42d1-b35e-c1f6306060d6@googlegroups.com> References: <784409a1-84f6-42d1-b35e-c1f6306060d6@googlegroups.com> Message-ID: <2135a3d0-02c0-47c0-9827-ccddbfb768f6@googlegroups.com> On Monday, February 16, 2015 at 12:35:00 PM UTC-6, Travis VanDame wrote: > I'm new to python and peewee and was looking for an example on how to query a mysql table with a datetime column only returning rows that are 30 days old. Well this is what I've come up with.... @classmethod def get_archive_xml(cls, day_count): return cls.select().where(cls.created + datetime.timedelta(days=int(day_count)) >= datetime.date.today()) From amirouche.boubekki at gmail.com Mon Feb 16 14:39:45 2015 From: amirouche.boubekki at gmail.com (Amirouche Boubekki) Date: Mon, 16 Feb 2015 19:39:45 +0000 Subject: A question about a list and subprocess.check_call() Message-ID: On Mon Feb 16 2015 at 7:40:42 PM Peter Otten <__peter__ at web.de> wrote: > David Aldrich wrote: > > > Hi Peter > > > > Thanks very much for your reply. I have added one more question below. > > > >> The straightforward approach is to pass a list or tuple: > >> > >> def build(build_options=()): > >> subprocess_check_call(("make",) + build_options) > >> > >> build(("flagA=true", "flagB=true")) > > > > This looks fine - I am trying it. > > > > I would like to display on the console the entire make command, so I have > > done this: > > > > def build(build_options=()): > > make_command = 'make '.join(map(build_options)) > > print('Build command: ' + make_command) > > subprocess.check_call(("make",)+build_options) > > > > but I get error: > > > > make_command = 'make '.join(map(build_options)) > > TypeError: map() must have at least two arguments. > > > > What would be the correct way to concatenate and display the elements in > > the tuple please? > > Hm, what do you expect map() to achieve? And "make " is not really the > string you want as a separator... > > You can join the strings in build_options with > > >>> build_options = "foo", "bar", "baz" > >>> " ".join(build_options) > 'foo bar baz' > >>> "make " + " ".join(build_options) > 'make foo bar baz' > > But you shouldn't do this as you have already learnt that spaces do not > separate arguments; in fact by default check_call() doesn't even use the > shell. An example to drive the point home: > > >>> from subprocess import check_call > >>> check_call(["python3", "-c", "import sys; print(sys.argv[1:])", "foo", > "bar"]) > ['foo', 'bar'] > 0 > >>> check_call(["python3", "-c", "import sys; print(sys.argv[1:])", "foo > bar"]) > ['foo bar'] > > Both invocations would appear to issue the same command in your debugging > print() call. list2cmdline() does a bit better; it resembles the command > issued if the shell were used: > > >>> print(subprocess.list2cmdline(["make"] + build_options)) > make foo "bar baz" > It's also possible to do it the other around using shlex.split. I prefer that version because I can easily copy/paste the command from code to the shell, it's also more readable IMO: > cmd = """python3 -O -c "import sys; print(sys.argv[1:])" foo bar "spam egg" """ > print(cmd) > subprocess.check_call(shlex.split(cmd)) shlex.split(cmd) must be prefered to cmd.split('') which doesn't take into account shell strings. paths with space must be wrapped in quote (talking of path, pathlib is awesome). https://docs.python.org/2.7/library/shlex.html?highlight=shlex#shlex.split Regards > > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndbecker2 at gmail.com Mon Feb 16 14:42:43 2015 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 16 Feb 2015 14:42:43 -0500 Subject: line_profiler: what am I doing wrong? References: Message-ID: Robert Kern wrote: > On 2015-02-13 13:35, Neal Becker wrote: >> Robert Kern wrote: >> >>> @profile >>> def run(): >>> pass >>> >>> run() >> >> No, this doesn't work either. Same failure >> >> kernprof -l test_prof.py >> Wrote profile results to test_prof.py.lprof >> Traceback (most recent call last): >> File "/home/nbecker/.local/bin/kernprof", line 9, in >> load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')() >> File "/home/nbecker/.local/lib/python2.7/site-packages/kernprof.py", line >> 221, >> in main >> execfile(script_file, ns, ns) >> File "test_prof.py", line 1, in >> @profile >> NameError: name 'profile' is not defined > > Ah, do you have the package `future` installed? > > https://github.com/rkern/line_profiler/issues/12 > Yes, I do. What do you suggest as a workaround? -- -- Those who don't understand recursion are doomed to repeat it From ethan at stoneleaf.us Mon Feb 16 14:49:00 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 16 Feb 2015 11:49:00 -0800 Subject: line_profiler: what am I doing wrong? In-Reply-To: References: Message-ID: <54E249AC.8030209@stoneleaf.us> On 02/16/2015 11:42 AM, Neal Becker wrote: > Robert Kern wrote: >> Ah, do you have the package `future` installed? >> >> https://github.com/rkern/line_profiler/issues/12 > > Yes, I do. What do you suggest as a workaround? I would think importing 'profile' directly would be the workaround. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From johnnybureddu at gmx.com Mon Feb 16 14:50:35 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Mon, 16 Feb 2015 11:50:35 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: <140bbc32-52df-41b6-a536-831e4d4a416e@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Mon Feb 16 14:51:55 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Mon, 16 Feb 2015 11:51:55 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: <7bd14589-336a-44d2-85c9-d005a1c1d8e4@googlegroups.com> -- redacted -- From jonathan.hayward at pobox.com Mon Feb 16 16:59:31 2015 From: jonathan.hayward at pobox.com (Jonathan Hayward) Date: Mon, 16 Feb 2015 15:59:31 -0600 Subject: Python implementations in JavaScript In-Reply-To: References: <54E0148D.7010008@stoneleaf.us> Message-ID: @all, thanks. I think I have Brython to try out first and others to maybe fall back on, which is the kind of information I wanted. Thanks, On Mon, Feb 16, 2015 at 3:59 PM, Jonathan Hayward < jonathan.hayward at pobox.com> wrote: > @all, thanks. I think I have Brython to try out first and others to maybe > fall back on, which is the kind of information I wanted. > > > Thanks, > > On Sat, Feb 14, 2015 at 9:37 PM, Ethan Furman wrote: > >> On 02/14/2015 05:28 PM, Ian Kelly wrote: >> > On Fri, Feb 13, 2015 at 4:05 PM, Jonathan Hayward wrote: >> >> >> >> What is the relative maturity of different Python implementations in >> JavaScript? Are any of the implementations ready to rely on? >> > >> > Brython is about two years old now. >> >> I remember a post from a few weeks ago that said Brython had worked just >> fine for their needs. >> >> -- >> ~Ethan~ >> >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> >> > > > -- > [image: Christos Jonathan Seth Hayward] > Jonathan S. Hayward, a full stack web developer with Python/Django and > AngularJS/jQuery. > > Articles on Technology ? Book: *Django > JavaScript Integration* > > ? *Email * ? Github ? > LinkedIn ? Main Website > ? *Professional Site * > > I invite you to visit my professional site! > -- [image: Christos Jonathan Seth Hayward] Jonathan S. Hayward, a full stack web developer with Python/Django and AngularJS/jQuery. Articles on Technology ? Book: *Django JavaScript Integration* ? *Email * ? Github ? LinkedIn ? Main Website ? *Professional Site * I invite you to visit my professional site! -------------- next part -------------- An HTML attachment was scrubbed... URL: From marfig at gmail.com Mon Feb 16 18:55:29 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Tue, 17 Feb 2015 00:55:29 +0100 Subject: Help setting up Sphinx Message-ID: Hello all, I'm at loss as to how properly setup sphinx to document my code. Getting a bit frustrated since I read everywhere it is an easy tool to use, but I'm either too dumb or I'm missing something very obvious. I setup sphinx with sphinx-quickstart. The following is how I answered the prompts: > Root path for the documentation [.]: docs > Separate source and build directories (y/n) [n]: > Name prefix for templates and static dir [_]: > Project name: SmUDGE > Author name(s): Mario Figueiredo > Project version: 0.1 > Project release [0.1]: > Source file suffix [.rst]: > Name of your master document (without suffix) [index]: > Do you want to use the epub builder (y/n) [n]: n > autodoc: automatically insert docstrings from modules (y/n) [n]: y > doctest: automatically test code snippets in doctest blocks (y/n) [n]: > intersphinx: link between Sphinx documentation of different projects (y/n) [n]: > todo: write "todo" entries that can be shown or hidden on build (y/n) [n]: > coverage: checks for documentation coverage (y/n) [n]: y > pngmath: include math, rendered as PNG images (y/n) [n]: > mathjax: include math, rendered in the browser by MathJax (y/n) [n]: > ifconfig: conditional inclusion of content based on config values (y/n) [n]: > viewcode: include links to the source code of documented Python objects (y/n) [n]: y > Create Makefile? (y/n) [y]: n > Create Windows command file? (y/n) [y]: y This resulted in a docs folder inside my project root, as intended. For clarity, here's the project folder structure: > /smudge --> project root > /smudge/docs --> sphinx based documentation > /smudge/smudge --> python scripts Next I edited /smudge/docs/conf.py to include the path to my source files: > sys.path.insert(0, os.path.abspath('../smudge/')) /smudge/smudge currently holds a single script file (item.py) with some classes and docstrings. Next I run $ make html to build the documentation, hoping the item.py file gets parsed and added to the docs build folder. But I get an empty documentation structure. docs/_build/html/index.html shows only three entries in the Indices and Tables section. Index Module Index Search Page Index is empty and Module Index is a 404 What am I missing in order to get item.py properly parsed by sphinx? From marfig at gmail.com Mon Feb 16 19:47:28 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Tue, 17 Feb 2015 01:47:28 +0100 Subject: Help setting up Sphinx References: Message-ID: On Tue, 17 Feb 2015 00:55:29 +0100, Mario Figueiredo wrote: > >What am I missing in order to get item.py properly parsed by sphinx? Nevermind. I found the problem. In order to document the source code it needs to be used in conjunction with sphinx-apidoc. From breamoreboy at yahoo.co.uk Mon Feb 16 21:05:12 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 17 Feb 2015 02:05:12 +0000 Subject: tempfile mktemp function Message-ID: This has been deprecated since 2.3 but is still in the 3.4 code and docs. This strikes me as weird in the extreme so can anybody explain why? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ken.hessel at gmail.com Mon Feb 16 21:08:22 2015 From: ken.hessel at gmail.com (ken.hessel at gmail.com) Date: Mon, 16 Feb 2015 18:08:22 -0800 (PST) Subject: Printing to printer, windows 7 Message-ID: Would seem to be a simple problem. I just want to print to my printer instead of the console using Python 2.7, Windows 7. Hours of looking through FAQ's and Google haven't yielded a solution. Any suggestions appreciated -- From skip.montanaro at gmail.com Mon Feb 16 21:24:58 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 16 Feb 2015 20:24:58 -0600 Subject: tempfile mktemp function In-Reply-To: References: Message-ID: I can't offer an explanation, but you might open an issue in the tracker proposing that it be removed in 3.5. If nothing else, you should get an authoritative-ish explanation for why it has yet to be deleted. Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.goldstick at gmail.com Mon Feb 16 21:26:23 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 16 Feb 2015 21:26:23 -0500 Subject: [Tutor] sql-like join on two lists or dictionaries In-Reply-To: References: <1423877077.197303.227364393.5C66E157@webmail.messagingengine.com> Message-ID: On Sun, Feb 15, 2015 at 2:14 PM, Peter Otten <__peter__ at web.de> wrote: > Joel Goldstick wrote: > > > You can dispense with the slicing if you use the str.split() method. It > > will put each item in a list. > > Only if there are no whitespace chars in the field. > > OT: > > Joel, your comments are already quoted when you first post them. Something > is broken in your workflow. > > not sure about quoting.. but the op didn't seem to have spaces in each > field > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Feb 16 22:02:33 2015 From: davea at davea.name (Dave Angel) Date: Mon, 16 Feb 2015 22:02:33 -0500 Subject: Printing to printer, windows 7 In-Reply-To: References: Message-ID: <54E2AF49.500@davea.name> On 02/16/2015 09:08 PM, ken.hessel at gmail.com wrote: > Would seem to be a simple problem. I just want to print to my printer instead of the console using Python 2.7, Windows 7. Hours of looking through FAQ's and Google haven't yielded a solution. Any suggestions appreciated -- > It is a simple problem. All you have to do is send the correct bytes to a file object that will transfer those bytes to the printer. It's been a long time since I used Windows, but if I recall, you just open the device "prn" or maybe "lpt1" and go. For example, to print "Hello world" to an Epson MX80, on a local parallel port, you'd do outfile = open("lpt1", "wb") print >> outfile, "Hello world" (To be polite, you should use "\\dev\\lpt1", but I don't know whether the simpler form has ever been deprecated by Windows) Unfortunately, printers have gotten increasingly complicated since the days of the MX80. For example on a HP laser printer, it won't start rendering till you send a hex "0c" to it (also known as a formfeed). You also may need to control fonts, sizes, justifications, etc., and that varies by printer model. Some printers want Postscript. So you are no longer talking to a printer device, but to a complex printer driver. I'm afraid when I want to print something for the last decade or so, I just create some form of document, and use the corresponding program to print it for me. For simple text, I put it in a simple text file, fire up emacs, and tell it to print. If you've managed to get the equivalent of a "print-to-file" output for your particular printer, then you can print it by just copying the bytes in that file to the "prn" or "lpt1" device. Depending on the data type you're trying to print, you may be able to control the external program from within Python. For example, if you're trying to print a pdf file, I got the following fragment from a google search: import win32api fname="C:\\somePDF.pdf" win32api.ShellExecute(0, "print", fname, None, ".", 0) In this fragment, you're asking the registered pdf viewer to print it for you. (presumably Acrobat reader or the equivalent) If you've got a Postscript printer, perhaps the following link will help, to print using PIL: http://effbot.org/imagingbook/introduction.htm The conventional way to print to an arbitrary printer, assuming it's already installed to your OS, is to use one of the GUI libraries. If your program is already GUI, then this is the way to go. For example, see tkinter, wxpython, qt, ... In this last case, the drivers and libraries abstract out the particular kind of printer, but you need a lot more software to use it. -- DaveA From davea at davea.name Mon Feb 16 22:09:03 2015 From: davea at davea.name (Dave Angel) Date: Mon, 16 Feb 2015 22:09:03 -0500 Subject: [Tutor] sql-like join on two lists or dictionaries In-Reply-To: References: <1423877077.197303.227364393.5C66E157@webmail.messagingengine.com> Message-ID: <54E2B0CF.4030400@davea.name> On 02/16/2015 09:26 PM, Joel Goldstick wrote: > On Sun, Feb 15, 2015 at 2:14 PM, Peter Otten <__peter__ at web.de> wrote: > >> Joel Goldstick wrote: >> >>> You can dispense with the slicing if you use the str.split() method. It >>> will put each item in a list. >> >> Only if there are no whitespace chars in the field. >> >> OT: >> >> Joel, your comments are already quoted when you first post them. Something >> is broken in your workflow. >> >> not sure about quoting.. but the op didn't seem to have spaces in each >> field Someone reading your message would figure you had typed nothing, since none of the lines start at the left margin. You're putting your new remarks after the ">" symbols, so they look like they're quoted from Peter. In most email programs, you just need to press once or twice to get an unindented line. Then type your response, so it's clear who wrote it. Worse, your message here is a reply to one originally posted in the tutor forum. -- DaveA From joel.goldstick at gmail.com Mon Feb 16 22:36:00 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 16 Feb 2015 22:36:00 -0500 Subject: [Tutor] sql-like join on two lists or dictionaries In-Reply-To: <54E2B0CF.4030400@davea.name> References: <1423877077.197303.227364393.5C66E157@webmail.messagingengine.com> <54E2B0CF.4030400@davea.name> Message-ID: On Mon, Feb 16, 2015 at 10:09 PM, Dave Angel wrote: > On 02/16/2015 09:26 PM, Joel Goldstick wrote: > >> On Sun, Feb 15, 2015 at 2:14 PM, Peter Otten <__peter__ at web.de> wrote: >> >> Joel Goldstick wrote: >>> >>> You can dispense with the slicing if you use the str.split() method. It >>>> will put each item in a list. >>>> >>> >>> Only if there are no whitespace chars in the field. >>> >>> OT: >>> >>> Joel, your comments are already quoted when you first post them. >>> Something >>> is broken in your workflow. >>> >>> not sure about quoting.. but the op didn't seem to have spaces in each >>> field >>> >> > > Someone reading your message would figure you had typed nothing, since > none of the lines start at the left margin. You're putting your new > remarks after the ">" symbols, so they look like they're quoted from Peter. > > In most email programs, you just need to press once or twice to > get an unindented line. Then type your response, so it's clear who wrote > it. > > Worse, your message here is a reply to one originally posted in the tutor > forum. > > -- > DaveA > -- > https://mail.python.org/mailman/listinfo/python-list > sorry all and DaveA.. not sure what went wrong -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Mon Feb 16 22:40:38 2015 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 17 Feb 2015 14:40:38 +1100 Subject: tempfile mktemp function In-Reply-To: References: Message-ID: <20150217034038.GA17961@cskk.homeip.net> On 16Feb2015 20:24, Skip Montanaro wrote: >I can't offer an explanation, but you might open an issue in the tracker >proposing that it be removed in 3.5. If nothing else, you should get an >authoritative-ish explanation for why it has yet to be deleted. It may be as simple as still being in system C libraries, so people are reluctant to break off access to it (while still discouraging it). Just conjecture, Cameron Simpson No electrons were harmed in the production of this message. - Dr. P. Gensheimer From nagle at animats.com Tue Feb 17 02:05:31 2015 From: nagle at animats.com (John Nagle) Date: Mon, 16 Feb 2015 23:05:31 -0800 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" Message-ID: Python 2.7.9, Windows 7 x64. (also 3.4.2 on Win7, and 3.4.0 on Ubuntu 14.04) There's something about the SSL cert for "https://www.verisign.com" that won't verify properly from Python. The current code looks like this: def testurlopen(host, certfile) : port = httplib.HTTPS_PORT sk = socket.socket(socket.AF_INET, socket.SOCK_STREAM) context = ssl.create_default_context(cafile=certfile) sock = context.wrap_socket(sk, server_hostname=host) try: sock.connect((host,port)) except EnvironmentError as message : print("Connection to \"%s\" failed: %s." % (host, message)) return False print("Connection to \"%s\" succeeded." % (host,)) return True Works for "python.org", "google.com", etc. I can connect to and dump the server's certificate for those sites. But for "verisign.com" and "www.verisign.com", I get Connection to "verisign.com" failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581). The certificate file, "cacert.pem", comes from Mozila's list of approved certificates, obtained from here: http://curl.haxx.se/ca/cacert.pem It has the cert for "VeriSign Class 3 Public Primary Certification Authority - G5" which is the root cert for "verisign.com". After loading that cert file into an SSL context, I can dump the context from Python with context.get_ca_certs() and get this dict for that cert: Cert: {'notBefore': u'Nov 8 00:00:00 2006 GMT', 'serialNumber': u'18DAD19E267DE8BB4A2158CDCC6B3B4A', 'notAfter': 'Jul 16 23:59:59 2036 GMT', 'version': 3L, 'subject': ((('countryName', u'US'),), (('organizationName', u'VeriSign, Inc.'),), (('organizationalUnitName', u'VeriSign Trust Network'),), (('organizationalUnitName', u'(c) 2006 VeriSign, Inc. - For authorized use only'),), (('commonName', u'VeriSign Class 3 Public Primary Certification Authority - G5'),)), 'issuer': ((('countryName', u'US '),), (('organizationName', u'VeriSign, Inc.'),), (('organizationalUnitName', u'VeriSign Trust Network'),), (('organizationalUnitName', u'(c) 2006 VeriSign, Inc. - For authorized use only'),), (('commonName', u'VeriSign Class 3 Public Primary Certification Authority - G5'),))} Firefox is happy with that cert. The serial number of the root cert matches the root cert Firefox displays. So the root cert file being used has the right cert for the cert chain back from "www.verisign.com". If I dump ssl.OPENSSL_VERSION from Python, I get "OpenSSL 1.0.1j 15 Oct 2014". That's an OK version. Something about that cert is unacceptable to the Python SSL module, but what? "CERTIFICATE VERIFY FAILED" doesn't tell me enough to diagnose the problem. John Nagle From lac at openend.se Tue Feb 17 03:00:40 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 17 Feb 2015 09:00:40 +0100 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: Message from John Nagle of "Mon, 16 Feb 2015 23:05:31 -0800." References: Message-ID: <201502170800.t1H80e6L028055@fido.openend.se> I've seen something like this: The requests module http://docs.python-requests.org/en/latest/ ships with its own set of certificates "cacert.pem" and ignores the system wide ones -- so, for instance, adding certificates to /etc/ssl/certs on your debian or ubuntu system won't work. I edited it by hand and then changed the REQUESTS_CA_BUNDLE environment variable to point to it. Perhaps your problem is along the same lines? Laura From David.Aldrich at EMEA.NEC.COM Tue Feb 17 04:32:03 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Tue, 17 Feb 2015 09:32:03 +0000 Subject: A question about a list and subprocess.check_call() In-Reply-To: References: Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B9421D6@EX10MBX02.EU.NEC.COM> >It's also possible to do it the other around using shlex.split. I prefer that version because >I can easily copy/paste the command from code to the shell, it's also more readable IMO: > cmd = """python3 -O -c "import sys; print(sys.argv[1:])" foo?bar "spam egg" """ > print(cmd) > subprocess.check_call(shlex.split(cmd)) Thanks again for the replies to my question. I especially like the solution of using shlex.split(). From falko.krause at gmail.com Tue Feb 17 05:40:11 2015 From: falko.krause at gmail.com (falko.krause at gmail.com) Date: Tue, 17 Feb 2015 02:40:11 -0800 (PST) Subject: Looking for a Python interface to TAPI on Win32 In-Reply-To: References: <2259b0e2.0307180401.5dae02f2@posting.google.com> <3f17f883$0$49107$e4fe514c@news.xs4all.nl> <2259b0e2.0307190529.57338b3f@posting.google.com> <2259b0e2.0307200604.44d343f4@posting.google.com> Message-ID: Am Freitag, 9. Januar 2004 00:31:27 UTC+1 schrieb Ray Chu: > I'd like to get familiar with the Win32 phone interface by using Python. Googling has just yielded Linux > apps. Would appreciate specific help or suggestions about where to look. > > - TIA > Ray I am also looking for answers, so far I found http://stackoverflow.com/questions/19426759/python-tapi-com-object-with-events and posted a question myself. http://stackoverflow.com/questions/28559614/how-do-i-get-a-tapi-3-event-incoming-call-in-python-or-vbs-usinig-the-com-inter If anybody could help that would be awesome From janhein.vanderburg at gmail.com Tue Feb 17 06:22:47 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Tue, 17 Feb 2015 03:22:47 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. Message-ID: In http://optarbvalintenc.blogspot.nl/ I propose a new way to encode arbitrarily valued integers and the python code that can be used as a reference for practical implementations of codecs. The encoding algorithm itself is optimized for transmission and storage requirements without any processor load consideration whatsoever. The next step is the development of the python code that minimizes processor requirements without compromising the algorithm. Is this the right forum to ask for help with this step or to obtain references to more suitable platforms? Thanks in advance, Jan-Hein. From rosuav at gmail.com Tue Feb 17 08:16:07 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 00:16:07 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On Tue, Feb 17, 2015 at 10:22 PM, wrote: > In http://optarbvalintenc.blogspot.nl/ I propose a new way to encode arbitrarily valued integers and the python code that can be used as a reference for practical implementations of codecs. > > The encoding algorithm itself is optimized for transmission and storage requirements without any processor load consideration whatsoever. > > The next step is the development of the python code that minimizes processor requirements without compromising the algorithm. > > Is this the right forum to ask for help with this step or to obtain references to more suitable platforms? This is a fine forum to ask in. However, you may find that the advice you get isn't quite what you were asking for. In my case, the advice I'm offering is: Don't do this. In over 99% of cases, the benefit from this kind of packed transmission format is negligible; it's much MUCH better to make your protocol simple readable text. Have a look at internet protocols like SMTP/POP/IMAP (the email trio), HTTP, FTP, and so on; all of them follow a basic pattern of connecting to a well-known port on a destination server, sending textual commands terminated by end-of-line, and receiving textual responses terminated by end-of-line. Most of the world wide web consists of HTTP requests (possibly using SSL/TLS, but that doesn't change the protocol), but even so, the advantage of packing the headers into a binary format just isn't worth the cost of making everything harder to debug. Because debugging is *hugely* easier when you have a text protocol. All you need to do is telnet or netcat to the appropriate port, type some commands, and eyeball the responses. I have a MUD client called Gypsum [1] which does that, with a few extra features, including (like netcat, but unlike telnet) listening on a port, so you can test a client; I've used it and its predecessor for testing myriad networking programs, both my own and other people's, to try to figure out what's going on. So if you want to develop a brand new protocol, here's what I'd suggest: 1) Mandate UTF-8 encoding everywhere 2) Stipulate \n as the end of line, and strip/ignore all \r found 3) Follow the basic model laid down by SMTP and POP: the server sends a greeting on startup, then everything's done with simple commands and responses. The server may also send unilateral messages, as long as they're unambiguously detectable (eg if it's notifying you of something that just happened). Sure, you might be able to pack your integers into something more compact than their decimal representations... but how much will you really gain? Most of the internet works on the basis of packets, and you'll find that the difference between a 200-byte packet and a 210-byte packet probably isn't even measurable; at very best, you might see an advantage when you transfer a huge file as a coherent blob, but that would mean dealing with a large number of these packetized integers. In the meantime, you make your protocol fragile and hard to read by eye, and you spend a lot of time developing your protocol, instead of just blatting simple text down the wire. Take the easy option; you can always make things more complicated later. [1] https://github.com/Rosuav/Gypsum ChrisA From davea at davea.name Tue Feb 17 09:12:56 2015 From: davea at davea.name (Dave Angel) Date: Tue, 17 Feb 2015 09:12:56 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: <54E34C68.6040700@davea.name> On 02/17/2015 06:22 AM, janhein.vanderburg at gmail.com wrote: > In http://optarbvalintenc.blogspot.nl/ I propose a new way to encode arbitrarily valued integers and the python code that can be used as a reference for practical implementations of codecs. > > The encoding algorithm itself is optimized for transmission and storage requirements without any processor load consideration whatsoever. > > The next step is the development of the python code that minimizes processor requirements without compromising the algorithm. > > Is this the right forum to ask for help with this step or to obtain references to more suitable platforms? > This is a fine forum for such a discussion. I for one would love to participate. However, note that it isn't necessary true that "the smaller the better" is a good algorithm. In context, there are frequently a number of tradeoffs, even ignoring processor time (as you imply). Many years ago I worked on some code that manipulated the intermediate files for a compiler. I had no say in the format, I just had to deal with it. They had a field type called a "compressed integer." It could vary between one byte and I think about six. And the theory was that it took less space than the equivalent format of fixed size integers. The catch from my point was that these integers could appear in the middle of a struct, and thus access to the later fields of the struct required a dynamic calculation. This put a huge onus on my code to read or write the data serially. I ended up solving it by writing code that generated 40 thousand lines of C++ header and body code, so that the rest of the code didn't care. Was it worth it? To reduce the size of some files that only lived a few seconds on disk? I seriously doubt it. But I learned a lot in the process. On another project, the goal was to be able to recover data from archives in spite of physical damage to some files. So I had to add selective redundancy for that. In the process, I also compress the data, but confine the compression algorithm to relatively small pieces of data, and label those pieces independently, so that any single decompression error affects only one unit of data. So going back to your problem, and assuming that the other issues are moot, what's your proposal? Are you compressing relative to a straight binary form of storage? Are you assuming anything about the relative likelihood of various values of integers? Do you provide anything to allow for the possibility that your prediction for probability distribution isn't met? -- DaveA From rosuav at gmail.com Tue Feb 17 09:34:47 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 01:34:47 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E34C68.6040700@davea.name> References: <54E34C68.6040700@davea.name> Message-ID: On Wed, Feb 18, 2015 at 1:12 AM, Dave Angel wrote: > They had a field type called a "compressed integer." It could vary between > one byte and I think about six. And the theory was that it took less space > than the equivalent format of fixed size integers. Oh, incidentally: If you want a decent binary format for variable-sized integer, check out the MIDI spec. Its varlen integer is simple: each byte carries seven bits of payload, and the high bit is set on all except the last byte. Anything under 128 is represented by the byte with that value; after that, you get a series of continuation bytes followed by a final byte. It could scale to infinity if you wanted to, though I think the MIDI spec doesn't have anything that can go quite that high, and it's a worst-case wastage of 12.5% (or, looking the other way, worst-case expansion of 14.2%), compared to an optimal eight-bit encoding. Given that MIDI often works with very small numbers, but occasionally could have to cope with much larger ones (eg "time to next event", which is often zero or very low, but once in a while could be anything at all), this is a good trade-off. If you know you're frequently going to work with numbers up to 2**31 but only occasionally larger than that, it might be better to take four bytes at a time, and use the high bit as a continuation bit, in the same way. It's all trade-offs. Decimal digits give you roughly 10/3 bits per character (== per byte, if you encode UTF-8). If you consider that your baseline, you can then try to justify your algorithm on the basis of "it's twice as efficient as decimal". Hexadecimal gives you 4 bits per character, at a relatively small cost; if you can't get better than about 6 bits per byte average throughput, I would say your algorithm is completely valueless - the difference will so rarely even matter, and you lose all the benefits I mentioned in my last email. I've tried to read through the original algorithm description, but I'm not entirely sure: How many payload bits per transmitted byte does it actually achieve? ChrisA From davea at davea.name Tue Feb 17 09:50:01 2015 From: davea at davea.name (Dave Angel) Date: Tue, 17 Feb 2015 09:50:01 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <54E34C68.6040700@davea.name> Message-ID: <54E35519.7060607@davea.name> On 02/17/2015 09:34 AM, Chris Angelico wrote: > On Wed, Feb 18, 2015 at 1:12 AM, Dave Angel wrote: >> They had a field type called a "compressed integer." It could vary between >> one byte and I think about six. And the theory was that it took less space >> than the equivalent format of fixed size integers. > > Oh, incidentally: If you want a decent binary format for > variable-sized integer, check out the MIDI spec. Its varlen integer is > simple: each byte carries seven bits of payload, and the high bit is > set on all except the last byte. Anything under 128 is represented by > the byte with that value; after that, you get a series of continuation > bytes followed by a final byte. It could scale to infinity if you > wanted to, though I think the MIDI spec doesn't have anything that can > go quite that high, and it's a worst-case wastage of 12.5% (or, > looking the other way, worst-case expansion of 14.2%), compared to an > optimal eight-bit encoding. Given that MIDI often works with very > small numbers, but occasionally could have to cope with much larger > ones (eg "time to next event", which is often zero or very low, but > once in a while could be anything at all), this is a good trade-off. > If you know you're frequently going to work with numbers up to 2**31 > but only occasionally larger than that, it might be better to take > four bytes at a time, and use the high bit as a continuation bit, in > the same way. It's all trade-offs. I have used that 7 bits/byte encoding myself for variable length integers. It worked well enough for its purpose. > > Decimal digits give you roughly 10/3 bits per character (== per byte, > if you encode UTF-8). If you consider that your baseline, you can then > try to justify your algorithm on the basis of "it's twice as efficient > as decimal". Hexadecimal gives you 4 bits per character, at a > relatively small cost; if you can't get better than about 6 bits per > byte average throughput, I would say your algorithm is completely > valueless - the difference will so rarely even matter, and you lose > all the benefits I mentioned in my last email. > > I've tried to read through the original algorithm description, but I'm > not entirely sure: How many payload bits per transmitted byte does it > actually achieve? Somehow when I first read the message, I missed the fact that there was a link to a web page with source code. But the first thing I'd expect to see would be a target estimate of the anticipated distribution of number values/magnitudes. For example, if a typical integer is 1500 bits, plus/minus 200 bits, I'd probably try encoding in base 65535, and have a terminator character of 0xffff. Without such a target, it's not a compression scheme, but an encoding one. An interesting point of history. At the time of Huffman's paper, it was provable that it was the best possible lossless compression. But there were some unwritten assumptions, such as that each character would be encoded with a whole number of bits. Changing that assumption makes arithmetic coding possible. Next assumption was that probabilities of each character didn't depend on context. Changing that assumption enables LZ. And so on. -- DaveA From rosuav at gmail.com Tue Feb 17 09:58:15 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 01:58:15 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E35519.7060607@davea.name> References: <54E34C68.6040700@davea.name> <54E35519.7060607@davea.name> Message-ID: On Wed, Feb 18, 2015 at 1:50 AM, Dave Angel wrote: > But the first thing I'd expect to see would be a target estimate of the > anticipated distribution of number values/magnitudes. For example, if a > typical integer is 1500 bits, plus/minus 200 bits, I'd probably try encoding > in base 65535, and have a terminator character of 0xffff. Sure. Not sure how you'd cope with an interior FFFF in the stream without drastically losing efficiency, though. > An interesting point of history. At the time of Huffman's paper, it was > provable that it was the best possible lossless compression. But there were > some unwritten assumptions, such as that each character would be encoded > with a whole number of bits. Changing that assumption makes arithmetic > coding possible. Next assumption was that probabilities of each character > didn't depend on context. Changing that assumption enables LZ. And so on. Didn't LZ predate arithmetic coding? But yes, removing those assumptions has enabled some pretty amazing alternatives. Just describing arithmetic coding to them is enough to blow most people's minds. (Especially my sister. She hates infinity, mainly - I think - because it doesn't fit inside her brain. I troll her occasionally with Hilbert's Hotel or anything involving the notion of different sizes of infinity.) ChrisA From davea at davea.name Tue Feb 17 10:18:33 2015 From: davea at davea.name (Dave Angel) Date: Tue, 17 Feb 2015 10:18:33 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <54E34C68.6040700@davea.name> <54E35519.7060607@davea.name> Message-ID: <54E35BC9.10101@davea.name> On 02/17/2015 09:58 AM, Chris Angelico wrote: > On Wed, Feb 18, 2015 at 1:50 AM, Dave Angel wrote: >> But the first thing I'd expect to see would be a target estimate of the >> anticipated distribution of number values/magnitudes. For example, if a >> typical integer is 1500 bits, plus/minus 200 bits, I'd probably try encoding >> in base 65535, and have a terminator character of 0xffff. > > Sure. Not sure how you'd cope with an interior FFFF in the stream > without drastically losing efficiency, though. That's why it was base 65535, not 65536. > >> An interesting point of history. At the time of Huffman's paper, it was >> provable that it was the best possible lossless compression. But there were >> some unwritten assumptions, such as that each character would be encoded >> with a whole number of bits. Changing that assumption makes arithmetic >> coding possible. Next assumption was that probabilities of each character >> didn't depend on context. Changing that assumption enables LZ. And so on. > > Didn't LZ predate arithmetic coding? I really don't know. I looked up Huffman's article (before Internet, I found it in a library) from the 1952 IRE journal (I believe that's where it was). I probably read it about 1980. But arithmetic coding and LZ both came later to me, and that's the order I saw them in. > But yes, removing those > assumptions has enabled some pretty amazing alternatives. Just > describing arithmetic coding to them is enough to blow most people's > minds. (Especially my sister. She hates infinity, mainly - I think - > because it doesn't fit inside her brain. I troll her occasionally with > Hilbert's Hotel or anything involving the notion of different sizes of > infinity.) -- DaveA From rosuav at gmail.com Tue Feb 17 10:25:19 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 02:25:19 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E35BC9.10101@davea.name> References: <54E34C68.6040700@davea.name> <54E35519.7060607@davea.name> <54E35BC9.10101@davea.name> Message-ID: On Wed, Feb 18, 2015 at 2:18 AM, Dave Angel wrote: >> Sure. Not sure how you'd cope with an interior FFFF in the stream >> without drastically losing efficiency, though. > > > That's why it was base 65535, not 65536. Doh. Yeah. I autocorrected in my head, but yes, base 65535 is safe. ChrisA From josef.achkar at gmail.com Tue Feb 17 11:38:46 2015 From: josef.achkar at gmail.com (Josef Achkar) Date: Tue, 17 Feb 2015 08:38:46 -0800 (PST) Subject: URL - Python ? Message-ID: <0ee84d3b-0223-4c0b-a089-1722504ed7bc@googlegroups.com> Hi i was wondering if it's possible to trace a website. If im for example is interested of the content of www.example.com/cars/audi and i have noticed that they are changing all URL's on the site so it dosent change at all and that it always stays the same like for example www.example.com and that i want to still be able to visit the page using "www.example.com/cars/audi" I would want to know if it's possible to trace the url to the new location From no.email at nospam.invalid Tue Feb 17 11:43:30 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 17 Feb 2015 08:43:30 -0800 Subject: python implementation of a new integer encoding algorithm. References: Message-ID: <87a90cfpzh.fsf@jester.gateway.pace.com> janhein.vanderburg at gmail.com writes: > The next step is the development of the python code that minimizes > processor requirements without compromising the algorithm. This is a reasonable place to ask specific python questions. The algorithm description itself is pretty confusing though, and it seems to address a problem that doesn't particularly seem to need a solution. It's pretty typical for applications needing compact representations to encode smallish variable-length integers in the VInt format, basically using 7 bits from each byte for data, and the 8th bit as a terminator flag. So VInt encodes numbers 0-127 in one byte, 128 to 16383 in 2 bytes, etc. Bignum applications generally use a length cell and a data array. From emil at fuse.pl Tue Feb 17 11:45:32 2015 From: emil at fuse.pl (Emil Oppeln-Bronikowski) Date: Tue, 17 Feb 2015 17:45:32 +0100 Subject: URL - Python ? In-Reply-To: <0ee84d3b-0223-4c0b-a089-1722504ed7bc@googlegroups.com> References: <0ee84d3b-0223-4c0b-a089-1722504ed7bc@googlegroups.com> Message-ID: <54E3702C.3080009@fuse.pl> > I would want to know if it's possible to trace the url to the new location If the webmaster did a proper thing you'll get server 3xx HTTP header that will point you to new URL. http://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection If you're using a good library to do your HTTP (like requests) the URL-following is build in and free >>>r = requests.get('http://github.com') >>>r.url 'https://github.com/' >>>r.status_code 200 >>>r.history [] If you want to track changes just check if response.history has some items in it. -- Emil Oppeln-Bronikowski *|* http://fuse.pl -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fusetiny.png Type: image/png Size: 2504 bytes Desc: not available URL: From johnnybureddu at gmx.com Tue Feb 17 13:16:16 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Tue, 17 Feb 2015 10:16:16 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From johnnybureddu at gmx.com Tue Feb 17 13:17:51 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Tue, 17 Feb 2015 10:17:51 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: -- redacted -- From johnnybureddu at gmx.com Tue Feb 17 13:19:34 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Tue, 17 Feb 2015 10:19:34 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From sjmsoft at gmail.com Tue Feb 17 15:10:43 2015 From: sjmsoft at gmail.com (sjmsoft at gmail.com) Date: Tue, 17 Feb 2015 12:10:43 -0800 (PST) Subject: Printing to printer, windows 7 In-Reply-To: References: Message-ID: <080efc88-1a16-4f88-85ca-828579795ffc@googlegroups.com> On Monday, February 16, 2015 at 10:08:33 PM UTC-4, ken.h... at gmail.com wrote: > Would seem to be a simple problem. I just want to print to my printer instead of the console using Python 2.7, Windows 7. Hours of looking through FAQ's and Google haven't yielded a solution. Any suggestions appreciated -- There is a good discussion on Tim Golden's site: http://timgolden.me.uk/python/win32_how_do_i/print.html As he notes, alas, there is a "disparity between the number and complexity of solutions and the simplicity of the requirement." HTH, Steve J. Martin From rgaddi at technologyhighland.invalid Tue Feb 17 15:52:27 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Tue, 17 Feb 2015 20:52:27 +0000 (UTC) Subject: Python & Peewee Query Example Needed References: <784409a1-84f6-42d1-b35e-c1f6306060d6@googlegroups.com> <2135a3d0-02c0-47c0-9827-ccddbfb768f6@googlegroups.com> Message-ID: On Mon, 16 Feb 2015 11:08:18 -0800, Travis VanDame wrote: > On Monday, February 16, 2015 at 12:35:00 PM UTC-6, Travis VanDame wrote: >> I'm new to python and peewee and was looking for an example on how to >> query a mysql table with a datetime column only returning rows that are >> 30 days old. > > Well this is what I've come up with.... > > @classmethod > def get_archive_xml(cls, day_count): > return cls.select().where(cls.created + > datetime.timedelta(days=int(day_count)) >= > datetime.date.today()) Without having actually executed your code, that looks about right. There's probably an efficiency pickup to be had only doing the datetime math once, but that's a technicality. Did it work? @classmethod def get_archive_xml(cls, day_count): then = datetime.date.today()-datetime.timedelta(days=int(day_count)) return cls.select().where(cls.created >= then) -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From c.candide at laposte.net Tue Feb 17 16:21:17 2015 From: c.candide at laposte.net (candide) Date: Tue, 17 Feb 2015 13:21:17 -0800 (PST) Subject: What the Pythons docs means by "container" ? Message-ID: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains : -------------------------- Some objects contain references to other objects; these are called containers. -------------------------- So containers contain : what a great definition! To be more precise, the "container" wording suggests a data structure _storing_ items somewhere and that the number of items in the container has a memory footprint. This is exacly the "container" definition given by the ISO C++ standard (cf. http://www.open-std.org/jtc1/sc22/open/n2356/lib-containers.html) : -------------------------- Containers are objects that store other objects. -------------------------- So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint in the contrary to the associated list : ---------------------------------------------- >>> from sys import getsizeof >>> r = range(10**6) >>> L = list(r) >>> getsizeof(r) 24 >>> getsizeof(L) 4500056 >>> ---------------------------------------------- Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container ! ---------------------------------------------- >>> from collections import Container >>> r = range(10**6) >>> isinstance(r, Container) True >>> ---------------------------------------------- So, what documentation means by the generic term of "container"? I agree with the existence of built-in containers (list, dict and the like) and the existence of special containers provided by the collections module (most of them inheriting from a built-in container) but I can't find a precise definition explaining what is a "container object" or a "container type". The docs at : https://docs.python.org/3.2/reference/datamodel.html#object.__contains__ explains that a container is an object compatible with the membership test (in and not in operators). So a file is a container ? recall a file supports the in operator : ---------------------------------------------- $ touch my_test.txt $ python3.2 Python 3.2.3 (default, Feb 28 2014, 00:22:33) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 42 in open("my_test.txt") False >>> from collections import Container >>> isinstance(open("my_test.txt"), Container) False >>> ---------------------------------------------- Reference of interest : http://blog.wachowicz.eu/?p=132 http://stackoverflow.com/questions/11575925/what-exactly-are-containers-in-python-and-what-are-all-the-python-container From cs at zip.com.au Tue Feb 17 16:39:54 2015 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 18 Feb 2015 08:39:54 +1100 Subject: What the Pythons docs means by "container" ? In-Reply-To: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: <20150217213954.GA91432@cskk.homeip.net> On 17Feb2015 13:21, candide wrote: >Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains : >-------------------------- >Some objects contain references to other objects; these are called containers. >-------------------------- And this is the basic thing, as you understand. >So containers contain : what a great definition! Shrug. It is a useful descriptive term when you want to talk about things. You might keep in mind that a "container"'s _primary_ purpose is usually its containing function. So a list. But not so some arbitrary object, even though most objects have references to other objects. [...] >So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint [...] >Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container ! You might find it duck types like other containers. [...] >The docs at : >https://docs.python.org/3.2/reference/datamodel.html#object.__contains__ >explains that a container is an object compatible with the membership test (in and not in operators). Duck typing again: this bases the term on behaviour, not memory use. >So a file is a container ? recall a file supports the in operator : Yes and no. A file can be iterated. "in" on a file reads the file as a side effect of iterating over it. A generator also supports "in", again by iteration. Neither of these things has a .__contains__ method; the "in" operator falls back to iteration if there is no .__contains__. A container usually lets you test "in" in a nondestructive/nonconsuming fashion. An iterable can be "in"ed, but it will be consumed. So iterables are not, of themselves, containers. So, range? A range object knows its bounds and state; it can answer "in" without consuming the range iteration, so it is effectively a container. Hoping this helps. Cheers, Cameron Simpson [Alain] had been looking at his dashboard, and had not seen me, so I ran into him. - Jean Alesi on his qualifying prang at Imola '93 From nagle at animats.com Tue Feb 17 17:57:35 2015 From: nagle at animats.com (John Nagle) Date: Tue, 17 Feb 2015 14:57:35 -0800 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: References: Message-ID: <54E3C75F.80307@animats.com> If I remove certs from my "cacert.pem" file passed to create_default_context, the Python test program rejects domains it will pass with the certs present. It's using that file. So that's not it. It seems to be an OpenSSL or cert file problem. I can reproduce the problem with the OpenSSL command line client: openssl s_client -connect www.verisign.com:443 -CAfile cacert.pem fails for "www.verisign.com", where "cacert.pem" has been extracted from Firefox's cert store. The error message from OpenSSL Verify return code: 20 (unable to get local issuer certificate) Try the same OpenSSL command for other domains ("google.com", "python.org") and no errors are reported. More later on this. So it's not a Python level issue. The only Python-specific problem is that the Python library doesn't pass detailed OpenSSL error codes through in exceptions. The Python exception text is "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581).", which is a generic message for most OpenSSL errors. John Nagle On 2/17/2015 12:00 AM, Laura Creighton wrote: > I've seen something like this: > > The requests module http://docs.python-requests.org/en/latest/ > ships with its own set of certificates "cacert.pem" > and ignores the system wide ones -- so, for instance, adding certificates > to /etc/ssl/certs on your debian or ubuntu system won't work. I edited > it by hand and then changed the REQUESTS_CA_BUNDLE environment variable > to point to it. > > Perhaps your problem is along the same lines? > > Laura > From nagle at animats.com Tue Feb 17 17:57:35 2015 From: nagle at animats.com (John Nagle) Date: Tue, 17 Feb 2015 14:57:35 -0800 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: References: Message-ID: <54E3C75F.80307@animats.com> If I remove certs from my "cacert.pem" file passed to create_default_context, the Python test program rejects domains it will pass with the certs present. It's using that file. So that's not it. It seems to be an OpenSSL or cert file problem. I can reproduce the problem with the OpenSSL command line client: openssl s_client -connect www.verisign.com:443 -CAfile cacert.pem fails for "www.verisign.com", where "cacert.pem" has been extracted from Firefox's cert store. The error message from OpenSSL Verify return code: 20 (unable to get local issuer certificate) Try the same OpenSSL command for other domains ("google.com", "python.org") and no errors are reported. More later on this. So it's not a Python level issue. The only Python-specific problem is that the Python library doesn't pass detailed OpenSSL error codes through in exceptions. The Python exception text is "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581).", which is a generic message for most OpenSSL errors. John Nagle On 2/17/2015 12:00 AM, Laura Creighton wrote: > I've seen something like this: > > The requests module http://docs.python-requests.org/en/latest/ > ships with its own set of certificates "cacert.pem" > and ignores the system wide ones -- so, for instance, adding certificates > to /etc/ssl/certs on your debian or ubuntu system won't work. I edited > it by hand and then changed the REQUESTS_CA_BUNDLE environment variable > to point to it. > > Perhaps your problem is along the same lines? > > Laura > From lac at openend.se Tue Feb 17 18:42:19 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 18 Feb 2015 00:42:19 +0100 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: Message from John Nagle of "Tue, 17 Feb 2015 14:57:35 -0800." <54E3C75F.80307@animats.com> References: <54E3C75F.80307@animats.com> Message-ID: <201502172342.t1HNgJYR027238@fido.openend.se> Possibly this bug? https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1014640 Laura From nagle at animats.com Tue Feb 17 19:28:50 2015 From: nagle at animats.com (John Nagle) Date: Tue, 17 Feb 2015 16:28:50 -0800 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: <201502172342.t1HNgJYR027238@fido.openend.se> References: <54E3C75F.80307@animats.com> <201502172342.t1HNgJYR027238@fido.openend.se> Message-ID: <54E3DCC2.6090704@animats.com> On 2/17/2015 3:42 PM, Laura Creighton wrote: > Possibly this bug? > https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1014640 > > Laura Probably that bug in OpenSSL. Some versions of OpenSSL are known to be broken for cases where there multiple valid certificate trees. Python ships with its own copy of OpenSSL on Windows. Tests for "www.verisign.com" Win7, x64: Python 2.7.9 with OpenSSL 1.0.1j 15 Oct 2014. FAIL Python 3.4.2 with OpenSSL 1.0.1i 6 Aug 2014. FAIL openssl s_client -OpenSSL 1.0.1h 5 Jun 2014 FAIL Ubuntu 14.04 LTS, using distro's versions of Python: Python 2.7.6 - test won't run, needs create_default_context Python 3.4.0 with OpenSSL 1.0.1f 6 Jan 2014. FAIL openssl s_client OpenSSL 1.0.1f 6 Jan 2014 PASS That's with the same cert file in all cases. The OpenSSL version for Python programs comes from ssl.OPENSSL_VERSION. The Linux situation has me puzzled. On Linux, Python is supposedly using the system version of OpenSSL. The versions match. Why do Python and the command line client disagree? Different options passed to OpenSSL by Python? Here's the little test program: http://www.animats.com/private/sslbug Please try that and let me know what happens on other platforms. Works with Python 2.7.9 or 3.x. John Nagle From c.candide at laposte.net Tue Feb 17 19:30:14 2015 From: c.candide at laposte.net (candide) Date: Tue, 17 Feb 2015 16:30:14 -0800 (PST) Subject: What the Pythons docs means by "container" ? In-Reply-To: References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> > > Shrug. It is a useful descriptive term when you want to talk about things. You Python Language Referenceb (PLR) is expected to give formal and acurate definition, not "descriptive term to talk about things". > might keep in mind that a "container"'s _primary_ purpose is usually its > containing function. So a list. But not so some arbitrary object, even though "containing function" ? Does PLR document this "containing function"? > most objects have references to other objects. have or contain ;) ? > Duck typing again: this bases the term on behaviour, not memory use. Again this is undocumented. Compare with the definition of duck-typed iterators, cf. https://docs.python.org/3.2/library/stdtypes.html#iterator-types On the other hand, recall that the new behavior of range since Python 3 is motivated by memory considerations, quoted from the doc : -------------------- The advantage of the range type over a regular list or tuple is that a range object will always take the same (small) amount of memory, no matter the size of the range it represents (as it only stores the start, stop and step values, calculating individual items and subranges as needed). -------------------- > Neither of these things has a .__contains__ method; the "in" operator falls > back to iteration if there is no .__contains__. The documentation is fuzzy and inconsistant here: containers are supposed to implement the membership test, dixit the doc. > > A container usually lets you test "in" in a nondestructive/nonconsuming "usually" ? I need something more acurate ;) > fashion. An iterable can be "in"ed, but it will be consumed. So iterables are > not, of themselves, containers. First this is undocumented. Second, range objects are considered as iterable by the doc. And you are telling us that an iterable is not a container. And range has a __contains__ method. > > So, range? A range object knows its bounds and state; it can answer "in" > without consuming the range iteration, so it is effectively a container. Again, this no-consuming feature is completely undocumented. And unconvincing: do you regard a linked list as not being a container ? From rosuav at gmail.com Tue Feb 17 19:49:46 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 11:49:46 +1100 Subject: What the Pythons docs means by "container" ? In-Reply-To: <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> Message-ID: On Wed, Feb 18, 2015 at 11:30 AM, candide wrote: > Python Language Referenceb (PLR) is expected to give formal and acurate definition, not "descriptive term to talk about things". In a lot of ways, technically accurate definitions can be useless, where sloppy but descriptive terms are useful. For instance, we can distinguish between an integer (doesn't contain things) and a list (does contain things), and then categorize sets, tuples, and SimpleNamespace()s with the latter, and exceptions, strings, and functions with the former. But technically, an exception has a reference to something else (a traceback), and could thus be said to contain it, and a function always has a reference to its global namespace (which itself contains that function), etc, so any specific and formal definition is likely to be quite useless. Also, a string can be said to "contain" a whole lot of substrings - "bcd" in "abcde" is True - even though those substrings don't actually have to exist as objects, so it's impossible for the string to retain references to those substrings. Even worse, a string always contains *itself*, which makes a definition based on the 'in' operator conflict entirely with a definition based on a real-world analogy involving a box with stuff in it. So, what's a container? It's a thing that you put other objects into. That's all the definition you really need, most of the time. If you want a better definition, you first need to ask yourself: What aspect of "containerness" matters here? Why do you need to distinguish container from non-container? If you want to know whether the 'in' operator can be used, then ask whether the 'in' operator can be used. If you want to know whether it's possible for this object to be part of a reference cycle, ask whether it's possible for this object to be part of a reference cycle. If you want to know whether this is a useful way to return multiple results from a function, then ... you get the idea. They're all very different aspects of containers, and not all containers have all those aspects. (For instance, a tuple can't [1] have a reference to itself. You can't get a reference cycle without having some other type of object, such as a list, involved.) "Is this a container?" isn't often a useful question to ask. ChrisA [1] You can use C code to create a tuple containing itself, but in C, you're most welcome to shoot yourself in the foot in other ways, too. From tjreedy at udel.edu Tue Feb 17 19:54:32 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Feb 2015 19:54:32 -0500 Subject: What the Pythons docs means by "container" ? In-Reply-To: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: On 2/17/2015 4:21 PM, candide wrote: > Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains : I use 'collection' rather than 'container'. > -------------------------- > Some objects contain references to other objects; these are called containers. > -------------------------- A mathematical set has 'members', like a club. Objects, like people, can be members of 0 to many clubs. Containing 'references', and the nature of said references is a computer implementation issue. There are no 'references' in math. -- Terry Jan Reedy From sohcahtoa82 at gmail.com Tue Feb 17 20:59:26 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Tue, 17 Feb 2015 17:59:26 -0800 (PST) Subject: Printing to printer, windows 7 In-Reply-To: References: Message-ID: On Monday, February 16, 2015 at 6:08:33 PM UTC-8, ken.h... at gmail.com wrote: > Would seem to be a simple problem. I just want to print to my printer instead of the console using Python 2.7, Windows 7. Hours of looking through FAQ's and Google haven't yielded a solution. Any suggestions appreciated -- I know that most HP printers support connecting on TCP port 9100 and sending plain text to be printed. Not sure about other brands. If you want to get fancy, you can look up PJL/PCL to do text formatting. Of course, this requires you to know the IP address of the printer. I wouldn't know how to get that programmatically. From python at mrabarnett.plus.com Tue Feb 17 21:11:50 2015 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 18 Feb 2015 02:11:50 +0000 Subject: Printing to printer, windows 7 In-Reply-To: <43r7eahbp697numc441je225t093dm7ce3@4ax.com> References: <54E2AF49.500@davea.name> <43r7eahbp697numc441je225t093dm7ce3@4ax.com> Message-ID: <54E3F4E6.4090405@mrabarnett.plus.com> On 2015-02-18 01:41, Dennis Lee Bieber wrote: > On Mon, 16 Feb 2015 22:02:33 -0500, Dave Angel declaimed > the following: > > >>It is a simple problem. All you have to do is send the correct bytes to >>a file object that will transfer those bytes to the printer. It's been >>a long time since I used Windows, but if I recall, you just open the >>device "prn" or maybe "lpt1" and go. >> > > C:\Users\Wulfraed\Documents>copy t.txt prn: > "prn:" is not a recognized device. > The system cannot find the file specified. > > C:\Users\Wulfraed\Documents>copy t.txt lpt1: > "lpt1:" is not a recognized device. > The system cannot find the file specified. > > And... > > C:\Users\Wulfraed\Documents>copy t.txt CN3BSFVGP805ZZ: > "CN3BSFVGP805ZZ:" is not a recognized device. > The system cannot find the file specified. > > That's the port name associated with my HP printer when looking at the > control panel properties. > > LPT1: appears to be linked to "Quicken PDF Printer" in the port list, > but is not available from command line. > You can use the "NET USE" command to make LPT1: refer to your printer. The details are on the Web. :-) From pascal.ortiz at gmail.com Tue Feb 17 21:14:44 2015 From: pascal.ortiz at gmail.com (candide) Date: Tue, 17 Feb 2015 18:14:44 -0800 (PST) Subject: What the Pythons docs means by "container" ? In-Reply-To: References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> Message-ID: <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> Le mercredi 18 f?vrier 2015 01:50:16 UTC+1, Chris Angelico a ?crit?: > > So, what's a container? It's a thing that you put other objects into. I agree with this approach. The important point to consider here is the last word in your definition : "into". There is the container and there is the content (the objects into). The so-called built-in containers (list, string, etc) are in conformance with this view. Now, regarding a range object as a container disagrees completely with the definition given in the PLR : there is no contents and hence there is no container. For instance, range(10**6) doesn't hold any kind of object, there are no reference to the int objects 0, 1, 2, ... As the range's docstring explains, range returns a VIRTUAL sequence. From kai.peters at gmail.com Tue Feb 17 23:07:09 2015 From: kai.peters at gmail.com (kai.peters at gmail.com) Date: Tue, 17 Feb 2015 20:07:09 -0800 (PST) Subject: Noob Parsing question Message-ID: Given data = '{[][]}' How can I efficiently get dictionaries for each of the data blocks framed by <> ? Thanks for any help KP From rosuav at gmail.com Tue Feb 17 23:16:05 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 15:16:05 +1100 Subject: Noob Parsing question In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 3:07 PM, wrote: > Given > > data = '{[][]}' > > How can I efficiently get dictionaries for each of the data blocks framed by <> ? > > Thanks for any help The question here is: What _can't_ happen? For instance, what happens if Fred's name contains a greater-than symbol, or a caret? If those absolutely cannot happen, your parser can be fairly straight-forward. Just put together some basic splitting (maybe a regex), and then split on the caret inside that. Otherwise, you may need a more stateful parser. ChrisA From kai.peters at gmail.com Tue Feb 17 23:35:01 2015 From: kai.peters at gmail.com (kai.peters at gmail.com) Date: Tue, 17 Feb 2015 20:35:01 -0800 (PST) Subject: Noob Parsing question In-Reply-To: References: Message-ID: > > Given > > > > data = '{[][]}' > > > > How can I efficiently get dictionaries for each of the data blocks framed by <> ? > > > > Thanks for any help > > The question here is: What _can't_ happen? For instance, what happens > if Fred's name contains a greater-than symbol, or a caret? > > If those absolutely cannot happen, your parser can be fairly > straight-forward. Just put together some basic splitting (maybe a > regex), and then split on the caret inside that. Otherwise, you may > need a more stateful parser. > > ChrisA The data string is guaranteed to be clean - no such irregularities occur. From rosuav at gmail.com Tue Feb 17 23:54:08 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 15:54:08 +1100 Subject: Noob Parsing question In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 3:35 PM, wrote: >> > Given >> > >> > data = '{[][]}' >> > >> > How can I efficiently get dictionaries for each of the data blocks framed by <> ? >> > >> > Thanks for any help >> >> The question here is: What _can't_ happen? For instance, what happens >> if Fred's name contains a greater-than symbol, or a caret? >> >> If those absolutely cannot happen, your parser can be fairly >> straight-forward. Just put together some basic splitting (maybe a >> regex), and then split on the caret inside that. Otherwise, you may >> need a more stateful parser. >> >> ChrisA > > The data string is guaranteed to be clean - no such irregularities occur. Okay! (Side point: You've stripped off all citations, here, so it's not clear who said what. My shorthand signature isn't as useful as the full line identifying date, time, and person. It's polite to keep those lines, at least for the first level of quoting.) What you want can be done with a regular expression. (Yes, yes, I know; now you have two problems.) >>> data = '{[][]}' >>> re.findall("<.*?>",data) ['', '', '', ''] >From there, you can crack open the different pieces: >>> for piece in re.findall("<.*?>",data): ... d = {} ... for elem in piece[1:-2].split("^"): ... key, value = elem.split("=",1) ... d[key] = value ... print(d) ... {'c': '45.22', 'b': 'Fred', 'a': '14'} {'b': 'Joe', 'a': '22'} {'c': '3.20', 'a': '17'} {'b': 'Soup', 'a': '72'} If you need some of those to be integers or floats, you'll need to do some post-processing on it, but this guarantees that you get the data out reliably. It depends on not having any of the special characters "=^<>" inside the elements, but other than that, it should be safe. ChrisA From frank at chagford.com Wed Feb 18 01:19:25 2015 From: frank at chagford.com (Frank Millman) Date: Wed, 18 Feb 2015 08:19:25 +0200 Subject: sqlite3 and dates Message-ID: Hi all sqlite3 does not have a DATE type, but the python module does a pretty good job of providing one - >>> import sqlite3 >>> conn = sqlite3.connect(':memory:', detect_types=sqlite3.PARSE_DECLTYPES) >>> cur = conn.cursor() >>> cur.execute('CREATE TABLE test (dob DATE)') >>> cur.execute('INSERT INTO TEST (dob) VALUES (?)', ('2015-03-31',)) >>> cur.execute('SELECT * FROM test') >>> cur.fetchone() (datetime.date(2015, 3, 31),) >>> However, the following does not return a date object - >>> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',)) >>> cur.fetchone() (2015,) >>> I don't know how easy this would be to implement, but it would be nice if it could be made to work. Is it worth filing a feature request? Frank Millman From rosuav at gmail.com Wed Feb 18 02:05:36 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 18:05:36 +1100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 5:19 PM, Frank Millman wrote: > However, the following does not return a date object - > >>>> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',)) > >>>> cur.fetchone() > (2015,) >>>> > > I don't know how easy this would be to implement, but it would be nice if it > could be made to work. Heh! Looks like the date is implemented as a slightly magical integer, so "cast to date" becomes "cast to integer" and you just get back 2015. Could be really easy to fix, could be nigh impossible... but sure, that seems a reasonable thing to ask for. Worst case, you get told it's not practical. But if you need more facilities than SQLite3 can offer, maybe it's time to move up to a full database server, instead of local files. Switching to PostgreSQL will give you all those kinds of features, plus a lot of other things that I would have thought pretty basic - like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't support that. ChrisA From breamoreboy at yahoo.co.uk Wed Feb 18 02:47:57 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 18 Feb 2015 07:47:57 +0000 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On 18/02/2015 06:19, Frank Millman wrote: > Hi all > > sqlite3 does not have a DATE type, but the python module does a pretty good > job of providing one - > >>>> import sqlite3 >>>> conn = sqlite3.connect(':memory:', detect_types=sqlite3.PARSE_DECLTYPES) >>>> cur = conn.cursor() >>>> cur.execute('CREATE TABLE test (dob DATE)') > >>>> cur.execute('INSERT INTO TEST (dob) VALUES (?)', ('2015-03-31',)) > >>>> cur.execute('SELECT * FROM test') > >>>> cur.fetchone() > (datetime.date(2015, 3, 31),) >>>> > > However, the following does not return a date object - > >>>> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',)) > >>>> cur.fetchone() > (2015,) >>>> > > I don't know how easy this would be to implement, but it would be nice if it > could be made to work. > > Is it worth filing a feature request? > > Frank Millman > Will this do? cur.execute('select current_date as "d [date]", current_timestamp as "ts [timestamp]"') row = cur.fetchone() print("current_date", row[0], type(row[0])) print("current_timestamp", row[1], type(row[1])) https://docs.python.org/3/library/sqlite3.html#default-adapters-and-converters -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From frank at chagford.com Wed Feb 18 02:49:10 2015 From: frank at chagford.com (Frank Millman) Date: Wed, 18 Feb 2015 09:49:10 +0200 Subject: sqlite3 and dates References: Message-ID: "Chris Angelico" wrote in message news:CAPTjJmrB+55CVgN6zTUawRf=rzn9LTaVi5TzhJSYvHHywK1vOw at mail.gmail.com... > On Wed, Feb 18, 2015 at 5:19 PM, Frank Millman wrote: >> However, the following does not return a date object - >> >>>>> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',)) >> >>>>> cur.fetchone() >> (2015,) >>>>> >> >> I don't know how easy this would be to implement, but it would be nice if >> it >> could be made to work. > > Heh! Looks like the date is implemented as a slightly magical integer, > so "cast to date" becomes "cast to integer" and you just get back > 2015. Could be really easy to fix, could be nigh impossible... but > sure, that seems a reasonable thing to ask for. Worst case, you get > told it's not practical. > > But if you need more facilities than SQLite3 can offer, maybe it's > time to move up to a full database server, instead of local files. > Switching to PostgreSQL will give you all those kinds of features, > plus a lot of other things that I would have thought pretty basic - > like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't > support that. > My accounting software supports three databases - MS Sql Server, PostgreSQL, and sqlite3. sqlite3 is not suitable for 'heavy-duty' applications, but it is ideal for demos and one-man businesses. Anyone can try out my software so long as they have python installed. They do not have to set up a database. sqlite3 does support ALTER TABLE, but with limited functionality. I think all you can do is add a column. Frank From lac at openend.se Wed Feb 18 02:49:47 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 18 Feb 2015 08:49:47 +0100 Subject: Python 2.7.9, 3.4.2 won't verify SSL cert for "verisign.com" In-Reply-To: Message from John Nagle of "Tue, 17 Feb 2015 16:28:50 -0800." <54E3DCC2.6090704@animats.com> References: <54E3C75F.80307@animats.com> <201502172342.t1HNgJYR027238@fido.openend.se><54E3DCC2.6090704@animats.com> Message-ID: <201502180749.t1I7nllj029635@fido.openend.se> I am away on a consulting gig, so I really only have my laptop to test on. Python 2.7.8 (default, Nov 18 2014, 14:57:17) debian version jessie/sid SSL test, with OpenSSL version OpenSSL 1.0.1j 15 Oct 2014. Connection to "verisign.com" failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581). Connection to "www.verisign.com" failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581). Connection to "python.org" succeeded. --------------------- Python 3.4.2 (default, Nov 13 2014, 07:01:52) debian version jessie/sid SSL test, with OpenSSL version OpenSSL 1.0.1j 15 Oct 2014. Connection to "verisign.com" failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581). Connection to "www.verisign.com" failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581). Connection to "python.org" succeeded. --------------- openssl s_client -connect www.verisign.com:443 -CAfile cacert.pem succeeds ..... So whatever problem there is, I have it too. Laura From frank at chagford.com Wed Feb 18 03:22:04 2015 From: frank at chagford.com (Frank Millman) Date: Wed, 18 Feb 2015 10:22:04 +0200 Subject: sqlite3 and dates References: Message-ID: "Mark Lawrence" wrote in message news:mc1g3n$q8j$1 at ger.gmane.org... > On 18/02/2015 06:19, Frank Millman wrote: >> Hi all >> >> sqlite3 does not have a DATE type, but the python module does a pretty >> good >> job of providing one - >> >> However, the following does not return a date object - >> >>>>> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',)) >> >>>>> cur.fetchone() >> (2015,) >>>>> >> > > Will this do? > > cur.execute('select current_date as "d [date]", current_timestamp as "ts > [timestamp]"') > row = cur.fetchone() > print("current_date", row[0], type(row[0])) > print("current_timestamp", row[1], type(row[1])) > I will have to experiment a bit, It looks as if it will do just fine. The magic incantation is 'detect_types=sqlite3.PARSE_DECLTYPES|PARSE_COLNAMES' I had not looked at PARSE_COLNAMES before. Very useful. Thanks, Mark Frank From janhein.vanderburg at gmail.com Wed Feb 18 03:55:26 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 00:55:26 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> On Tuesday, February 17, 2015 at 2:17:02 PM UTC+1, Chris Angelico wrote: > This is a fine forum to ask in. However, you may find that the advice > you get isn't quite what you were asking for. In my case, the advice > I'm offering is: Don't do this. Thanks Chris; let me explain why I want this. As you rightly point out, human readable encoding enables debugging without the need for specialized tools that assemble/disassemble character streams. I also agree that saving space in a partition of a character stream like a communication packet or persistent memory sector won't do much good if it does not save additional partitions. The human centered encoding versus machine oriented encoding discussion dates from some decades ago now, and I am personally still not convinced that human readable text is all that sensible in most applications. The main problem I see is that we are trying to make computers behave like human beings, and that introduces interpretation risks and resource requirements that should be avoided. I'll be happy to discuss this subject again, but let's do that in a different thread. The difference between two encoded stream lengths within a single partition may not be relevant, but the difference between needing 1 or 2 partitions is quite significant. Especially in applications that depend on very limited bandwidth communication channels. Since all messages in my applications are primarily composed of integer values, finding the algorithm that fundamentally minimizes the number of bits to encode such a value will increase my chances to save partitions needed to encode the message, even without adding compression techniques at the message level. Note that I don't assume any properties of the integer values to be encoded. > Take the > easy option; you can always make things more complicated later. That makes sense alright. No offense, but I still believe that human readable text encoding complicates things right now and shouldn't be tried until "my way" has proven unpractical in its first application. Consider it to be a theoretical challenge: how do I find the general encoding of an arbitrary integer value that minimizes the number of bits needed and given that algorithm, find the python code that minimizes the processor load inflicted by the codec implementation. From janhein.vanderburg at gmail.com Wed Feb 18 03:59:51 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 00:59:51 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> On Tuesday, February 17, 2015 at 3:13:41 PM UTC+1, Dave Angel wrote: > This is a fine forum for such a discussion. I for one would love to > participate. However, note that it isn't necessary true that "the > smaller the better" is a good algorithm. In context, there are > frequently a number of tradeoffs, even ignoring processor time (as you > imply). Thanks Dave; about those trade offs: I agree that allowing variable length encoding in general forces the programmer to process records in a file sequentially. Random access to individual data items in such a stream is simply not an option, unless all items have been indexed in a directory that accompanies the data character stream. So if you need random access to individual items put limits to each individual data item. My applications do not allow me to do that, because I would always be to restrictive and be caught by the "640k should be enough for any programmer" trap. > So going back to your problem, and assuming that the other issues are > moot, what's your proposal? Are you compressing relative to a straight > binary form of storage? Are you assuming anything about the relative > likelihood of various values of integers? Do you provide anything to > allow for the possibility that your prediction for probability > distribution isn't met? I'm not compressing sequences of integer encodings but encoding individual integers optimally without any assumptions about their values. From janhein.vanderburg at gmail.com Wed Feb 18 04:04:16 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 01:04:16 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <54E34C68.6040700@davea.name> Message-ID: <65b209a0-33fc-4c1f-8af6-de3a4626623c@googlegroups.com> On Tuesday, February 17, 2015 at 3:35:16 PM UTC+1, Chris Angelico wrote: > Oh, incidentally: If you want a decent binary format for > variable-sized integer, check out the MIDI spec. I did some time ago, thanks, and it is indeed a decent format. I also looked at variations of that approach. None of them beats "my" concept of two counters that cooperatively specify field lengths and represented integer values. > I've tried to read through the original algorithm description, but I'm > not entirely sure: How many payload bits per transmitted byte does it > actually achieve? I don't think that payload bits per byte makes sense in this concept. From janhein.vanderburg at gmail.com Wed Feb 18 04:06:32 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 01:06:32 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <87a90cfpzh.fsf@jester.gateway.pace.com> References: <87a90cfpzh.fsf@jester.gateway.pace.com> Message-ID: On Tuesday, February 17, 2015 at 5:43:43 PM UTC+1, Paul Rubin wrote: > This is a reasonable place to ask specific python questions. The > algorithm description itself is pretty confusing though, and it seems to > address a problem that doesn't particularly seem to need a solution. > It's pretty typical for applications needing compact representations to > encode smallish variable-length integers in the VInt format, basically > using 7 bits from each byte for data, and the 8th bit as a terminator > flag. So VInt encodes numbers 0-127 in one byte, 128 to 16383 in 2 > bytes, etc. Bignum applications generally use a length cell and a data > array. Thanks Paul, The algorithm sort of combines the Vint and Bignum approaches. I'm sorry to read that my description is confusing; is the illustration with 2 bit characters helpful or should it be extended? From rosuav at gmail.com Wed Feb 18 04:36:10 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 20:36:10 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> References: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> Message-ID: On Wed, Feb 18, 2015 at 7:55 PM, wrote: >> Take the >> easy option; you can always make things more complicated later. > That makes sense alright. > No offense, but I still believe that human readable text encoding complicates things right now and shouldn't be tried until "my way" has proven unpractical in its first application. > Consider it to be a theoretical challenge: how do I find the general encoding of an arbitrary integer value that minimizes the number of bits needed and given that algorithm, find the python code that minimizes the processor load inflicted by the codec implementation. > I would actually look at it the other way: a human-readable encoding is the easy way (you can simply print() your numbers and int() them on the way back in), and until you can prove that it's impractical, don't write a single line of code towards this algorithm. But let's get some figures, though: How many payload bits per byte can you achieve? What's your average going to be? You have two baselines to beat: 4 bits per byte (hexadecimal), and 7 bits per byte (MIDI varlen, or whatever you want to call it; I first met it with MIDI, but it's been used in so many places that other people have other names for it). ChrisA From rosuav at gmail.com Wed Feb 18 04:53:26 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 20:53:26 +1100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman wrote: > My accounting software supports three databases - MS Sql Server, PostgreSQL, > and sqlite3. > > sqlite3 is not suitable for 'heavy-duty' applications, but it is ideal for > demos and one-man businesses. Anyone can try out my software so long as they > have python installed. They do not have to set up a database. I wouldn't trust sqlite3 with my accounting... but then, I'm becoming more and more disillusioned with most full-scale accounting packages, too. I'm seriously looking toward a git-managed directory of text files as being the accounting package of the future; a few pre-commit hooks (and/or pre-receive if you want to have a central repo that people push to) to check file formats and stuff; and then some scripts that do maintenance and reporting (eg updating inventory quantities when you mark an invoice as completed). I want to see a big company work this way... it's clearly not a problem for a one-man operation, as I've been doing exactly that for some time (with very few scripts), and we do know that git scales to global operations just fine, but will it actually work? Inquiring minds must know! But anyway, I wouldn't push people onto sqlite3 for anything serious, mainly because it sucks at concurrency, secondarily because it lacks a number of common features. I'd push people to PostgreSQL. > sqlite3 does support ALTER TABLE, but with limited functionality. I think > all you can do is add a column. Ah, true. Minor support for altering tables, not quite none. But still, there's a lot of limitations that will bite you badly any time you try to migrate a schema using anything other than the brute-force "create entire new database and import the content". ChrisA From lac at openend.se Wed Feb 18 05:32:42 2015 From: lac at openend.se (Laura Creighton) Date: Wed, 18 Feb 2015 11:32:42 +0100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: Message from Chris Angelico of "Wed, 18 Feb 2015 20:36:10 +1100." References: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> Message-ID: <201502181032.t1IAWg2S031162@fido.openend.se> Hi Jan. I'm an old fart. In the late 1970s, when I started programming these things, and memory was non-existant, we came up with all sorts of data compression algorithms which were absolutely necessary to get any work done whatsoever. Should you ever need an assembler programmer for quick and dirty hacks for the PDP-11 line (11/20 and 11/05 preferred as it is harder) I am still the woman for the job. Indeed, I spent most of my 20s finding better and better ways to fit programs into smaller and smaller memory footprints. I perfectly understand the intellectual thrill of doing such things. As puzzles go, it is about as cool a one as exists, and its all for things that matter -- for real. However, in the matter of financial compensation and world recognition, you have just laid a very large goose-egg. The VAX-11/780 was introduced on October 25, 1977, according to wikipedia. But in my world, it was 1982 before I got to see the first one. And it was godly more expensive than a pdp-11, but the writing was on the wall. The thing could page, and so all the techniques we learned for making our code concise -- let alone the dirty tricks I specialised in -- were no longer relevant. >From the mid 1980s onward I have been telling people 'your code is ugly, please tighten it up by refactoring it and ' and when I am their instructor they grumble and do it, and otherwise they flip me the bird. In their eyes, it doesn't matter how the code _looks_ as long as it does the job. And I deeply sympathise. But what I am going for is not a 'death - by looking unfashionable' but rather a demand that good code is clear to understand. Because what I have learned, that Brian Kernighan expressed a long time ago is that: Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Your proposed encoding scheme (if it does as you say, I have not analysed it) scores very, very high in the _cleverness_ department. Enough that a lot of people, who aren't as clever as you do, have no hope in hell of ever being clever enough to debug something that uses it. Therefore, you will never see widespread adoption of your scheme, no matter how brilliantly it does as you say, because we all need things that are easier to debug more than we need better compression. So now you are sad. I was sad, too, but the sooner I learned this the sooner I could stop wasting my time creating algorithms that provided cool functionality that people hated for the same reasons I found them cool. You need to find a different sort of algorithm that people like to use if you want to get widespread success in the world of widely used algorithms. If you have found a way to improve on Lemel-Ziv, then this will count. But it may be that your next step is 'how to encode things that are not phonetic language'. Go look -- for the next few months -- at how MIDI stores sounds. You will find plenty of places for improvement, but the idea is not to improve the standard but to learn it well enough that you can see things in the non-alphabetic world. So then, now what? If you are still fired up with the desire to compress things, then there is a huge, _very well paying_ market I want to introduce you to. And this is _tech support for porn sites_. Porn sites make a ton of money, indeed the numbers are scary. And here the idea of 'I saved 2% of time/bandwidth/disk space/' really matters. You can really save money for them, and it really matters to them. Since I have never found sex 'dirty' and indeed consider it one of the great joys in life, I have never found anything wrong with working for porn sites. And, hell, out of male porn Jimmy Wales made wikipedia. Haven't we all benefitted? But, right now, you are, alas for you, about 45 years too late for the ideas you are sprouting. I had similar ones about 30 years too late and, well, they only worked for me for about 3-5 years. Sucks to be you, friend -- you needed to be your grandfather, I fear. Laura Creighton From ben+python at benfinney.id.au Wed Feb 18 05:57:57 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 18 Feb 2015 21:57:57 +1100 Subject: People hated it for the same reasons I found them cool (was: python implementation of a new integer encoding algorithm.) References: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> <201502181032.t1IAWg2S031162@fido.openend.se> Message-ID: <857fvfbi6i.fsf_-_@benfinney.id.au> Laura Creighton writes: > So now you are sad. I was sad, too, but the sooner I learned this the > sooner I could stop wasting my time creating algorithms that provided > cool functionality that people hated for the same reasons I found them > cool. +1 QotW -- \ ?Human reason is snatching everything to itself, leaving | `\ nothing for faith.? ?Bernard of Clairvaux, 1090?1153 CE | _o__) | Ben Finney From dfnsonfsduifb at gmx.de Wed Feb 18 06:11:45 2015 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Wed, 18 Feb 2015 12:11:45 +0100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On 18.02.2015 08:05, Chris Angelico wrote: > But if you need more facilities than SQLite3 can offer, maybe it's > time to move up to a full database server, instead of local files. > Switching to PostgreSQL will give you all those kinds of features, > plus a lot of other things that I would have thought pretty basic - > like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't > support that. I see you're running a lawnmower. Maybe you should switch to a combine harvester. That'll get you extra features like a reciprocating knife cutter bar. I was quite surprised that regular lawnmowers don't support those. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht ?ffentlich! Ah, der neueste und bis heute genialste Streich unsere gro?en Kosmologen: Die Geheim-Vorhersage. - Karl Kaos ?ber R?diger Thomas in dsa From rosuav at gmail.com Wed Feb 18 06:21:35 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 22:21:35 +1100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer wrote: > On 18.02.2015 08:05, Chris Angelico wrote: > >> But if you need more facilities than SQLite3 can offer, maybe it's >> time to move up to a full database server, instead of local files. >> Switching to PostgreSQL will give you all those kinds of features, >> plus a lot of other things that I would have thought pretty basic - >> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >> support that. > > I see you're running a lawnmower. Maybe you should switch to a combine > harvester. That'll get you extra features like a reciprocating knife > cutter bar. I was quite surprised that regular lawnmowers don't support > those. SQLite3 is fine for something that's basically just a more structured version of a flat file. You assume that nobody but you has the file open, and you manipulate it just the same as if it were a big fat blob of JSON, but thanks to SQLite, you don't have to rewrite the whole file every time you make a small change. That's fine. But it's the wrong tool for any job involving multiple users over a network, and quite probably the wrong tool for a lot of other jobs too. It's the smallest-end piece of software that can truly be called a database. I would consider it to be the wrong database for serious accounting work, and that's based on the ranting of a majorly-annoyed accountant who had to deal with issues in professional systems that had made similar choices in back-end selection. You're welcome to disagree, but since PostgreSQL doesn't cost any money and (on Linux at least; can't speak for other platforms) doesn't take significant effort to set up, I will continue to recommend it. ChrisA From dfnsonfsduifb at gmx.de Wed Feb 18 06:57:36 2015 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Wed, 18 Feb 2015 12:57:36 +0100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On 18.02.2015 12:21, Chris Angelico wrote: > SQLite3 is fine for something that's basically just a more structured > version of a flat file. You assume that nobody but you has the file > open, and you manipulate it just the same as if it were a big fat blob > of JSON, but thanks to SQLite, you don't have to rewrite the whole > file every time you make a small change. That's fine. But it's the > wrong tool for any job involving multiple users over a network, and > quite probably the wrong tool for a lot of other jobs too. Your assessment that some tools fit certain problems and don't fit different problems is entirely correct. SQLite does the job that it is supposed to do and it fills that nieche well. > It's the > smallest-end piece of software that can truly be called a database. I > would consider it to be the wrong database for serious accounting > work, and that's based on the ranting of a majorly-annoyed accountant > who had to deal with issues in professional systems that had made > similar choices in back-end selection. It probably is the wrong database for serious accounting work, and it's probably also the wrong database for doing multivariate statistical analysis on sparse matrices that you store in tables. You could similarly argue that a hammer is the wrong tool to drive in a screw and you'd be correct in that assessment. But it's completely besides the point. SQLite and Postgres are so vastly different in their setup, configuration, capabilities and requirements that the original developer has to have done a MAJOR error in judgement so that a change from one to the other would not be ill-advised. > You're welcome to disagree, but since PostgreSQL doesn't cost any > money and (on Linux at least; can't speak for other platforms) doesn't > take significant effort to set up, I will continue to recommend it. I work with Postgres on a professional, day-to-day basis. And while it's free, it *does* take a significant effort to setup and it *does* take a significant effort to maintain. Especially in comparison with something like SQLite that literally has no setup at all. PostgreSQL is great. It's an incredible database and that it's free is amazing. But in very few settings will it be a replacement for SQLite. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht ?ffentlich! Ah, der neueste und bis heute genialste Streich unsere gro?en Kosmologen: Die Geheim-Vorhersage. - Karl Kaos ?ber R?diger Thomas in dsa From samuelmarks at gmail.com Wed Feb 18 07:08:55 2015 From: samuelmarks at gmail.com (Samuel Marks) Date: Wed, 18 Feb 2015 04:08:55 -0800 (PST) Subject: Dependencies from git aren't being installed? (setuptools, pip) Message-ID: <04153a2a-e853-4d6e-9a6d-ede80e8d58ca@googlegroups.com> I'm sure I've just misconfigured my setup.py, requirements.txt and/or project structure, but I've tried: * install_requires * requirements.txt * install_requires + dependency_links With (for install_requires): * bettertutors_sql_models==dev * bettertutors_sql_models For requirements.txt, dependency_links: * git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models * git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models * https://github.com/bettertutors/sql-models/archive/master.zip#egg=bettertutors_sql_models * https://github.com/bettertutors/sql-models/archive/master.zip * https://github.com/bettertutors/sql-models * https://github.com/bettertutors/sql-models/tarball/master#egg=sql-models Also asked here: http://stackoverflow.com/q/28540839 At one point I even tried writing my own install_deps function() to be run before setup(): https://gist.github.com/SamuelMarks/45a998a83dd60ddbadbc But nothing--apart from `cd`ing to my sql-models dir and running `pip install .`--is working for me. I'm using Python 2.7.*, and have experienced identical issues on Windows and Linux (with/without virtualenv). How am I meant to write in this dependency? From rosuav at gmail.com Wed Feb 18 07:13:11 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 23:13:11 +1100 Subject: Issues pip-installing pywin32 Message-ID: (Before I begin, I want to make it clear that I can still go poke around on SourceForge and grab the appropriate installer [1], and that does work. But if I'm going to tell someone else how to set up this program, I'd much rather be able to recommend pip.) On a fresh Python 3.4 running on Windows 7, attempting to "pip install pywin32" doesn't work: C:\Users\Rosuav>pip install pywin32 Downloading/unpacking pywin32 Could not find any downloads that satisfy the requirement pywin32 Some externally hosted files were ignored (use --allow-external pywin32 to allow). Cleaning up... No distributions at all found for pywin32 Storing debug log for failure in C:\Users\Rosuav\pip\pip.log C:\Users\Rosuav>pip install --allow-external pywin32 You must give at least one requirement to install (see "pip help install") Okay, so the obvious command doesn't work, and gives a not-particularly-helpful error message, but I know how to Google, I can get past this. C:\Users\Rosuav>pip install --allow-external pywin32 --allow-unverified pywin32 pywin32 Downloading/unpacking pywin32 Could not find any downloads that satisfy the requirement pywin32 Cleaning up... No distributions at all found for pywin32 Storing debug log for failure in C:\Users\Rosuav\pip\pip.log Huh. The PyPI categorization seems to exclude Python 3.4 support (despite versions for both 3.4 and 3.5 existing on the sourceforge page). Let's try that with 2.7, just to see what happens. (I could alternatively backlevel to 3.3, I suppose.) C:\Users\Rosuav>\python27\python -m pip --version pip 1.5.6 from C:\python27\lib\site-packages (python 2.7) C:\Users\Rosuav>\python27\python -m pip install --allow-external pywin32 --allow-unverified pywin32 pywin32 Downloading/unpacking pywin32 Could not find any downloads that satisfy the requirement pywin32 Cleaning up... No distributions at all found for pywin32 Storing debug log for failure in C:\Users\Rosuav\pip\pip.log The log contains these lines, which may be a clue as to what's going on... Could not fetch URL https://sourceforge.net/projects/pywin32/files/pywin32/ (from https://pypi.python.org/simple/pywin32/): connection error: hostname 'sourceforge.net' doesn't match either of 'cloudfront.net', '*.cloudfront.net' Will skip URL https://sourceforge.net/projects/pywin32/files/pywin32/ when looking for download links for pywin32 Could not find any downloads that satisfy the requirement pywin32 Does this mean there's an HTTPS configuration error at sourceforge? And if that is indeed the problem, is there any way that I, as an end user, can bypass the check? My alternative is using a non-SSL link, so it's not like I'd actually have worse security. At this point, I am REALLY glad my Dad asked me to set this all up for him, rather than just giving him some instructions and saying "go for it". (And to think, all I want is a simple program to bounce some info out from a Windows VM onto the Linux host. So much hassle for something so simple.) [1] http://sourceforge.net/projects/pywin32/files/pywin32/ From rosuav at gmail.com Wed Feb 18 07:14:32 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Feb 2015 23:14:32 +1100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer wrote: > SQLite and Postgres are so vastly different in their setup, > configuration, capabilities and requirements that the original developer > has to have done a MAJOR error in judgement so that a change from one to > the other would not be ill-advised. On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman wrote: > My accounting software supports three databases - MS Sql Server, PostgreSQL, > and sqlite3. Johannes, are you saying that Frank made three major errors of judgement? :) ChrisA From dfnsonfsduifb at gmx.de Wed Feb 18 08:13:01 2015 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Wed, 18 Feb 2015 14:13:01 +0100 Subject: sqlite3 and dates In-Reply-To: References: Message-ID: On 18.02.2015 13:14, Chris Angelico wrote: > On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer wrote: >> SQLite and Postgres are so vastly different in their setup, >> configuration, capabilities and requirements that the original developer >> has to have done a MAJOR error in judgement so that a change from one to >> the other would not be ill-advised. > > On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman wrote: >> My accounting software supports three databases - MS Sql Server, PostgreSQL, >> and sqlite3. > > Johannes, are you saying that Frank made three major errors of judgement? :) I'm totally pulling my fifth! :-P Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht ?ffentlich! Ah, der neueste und bis heute genialste Streich unsere gro?en Kosmologen: Die Geheim-Vorhersage. - Karl Kaos ?ber R?diger Thomas in dsa From lynto28 at gmail.com Wed Feb 18 08:49:17 2015 From: lynto28 at gmail.com (Didymus) Date: Wed, 18 Feb 2015 05:49:17 -0800 (PST) Subject: Logging with Custom Levels not working Message-ID: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> Greetings, I've been setting up a custom logging for a project here and got some great examples of code from stackoverflow. The only problem I'm having is that no matter what I set the log level too all the messages print out... Verbosity Level set to: 55 [PVERBOSE] Performance Verbose Level [PDEBUG] Performance Debug Level [PMESSAGE] Performance Message Level [PWARNING] Performance Warning Level [PERROR] Performance Error Level This should only print the PERROR level, but everything in the custom levels is being printed out. Not sure what I'm doing wrong here. Any help is greatly appreciated. We are use Python 2.7.5 .Example Code below: Thanks Tom #!/usr/bin/python -tt import logging # # Performance Custom Message Levels 'PERROR' : 55, 'PWARNING' : 54, 'PMESSAGE': 53, 'PDEBUG' : 52, 'PVERBOSE' : 51 # PERROR_NUM = 55 PWARNING_NUM = 54 PMESSAGE_NUM = 53 PDEBUG_NUM = 52 PVERBOSE_NUM = 51 # Performance Error... logging.addLevelName(PERROR_NUM, "PERROR") def perror(self, message, *args, **kws): """ Performance Error Message Level """ # Yes, logger takes its '*args' as 'args'. self._log(PERROR_NUM, message, args, **kws) logging.Logger.perror = perror # Performance Warning... logging.addLevelName(PWARNING_NUM, "PWARNING") def pwarning(self, message, *args, **kws): """ Performance Warning Message Level """ # Yes, logger takes its '*args' as 'args'. self._log(PWARNING_NUM, message, args, **kws) logging.Logger.pwarning = pwarning # Performance Messages... logging.addLevelName(PMESSAGE_NUM, "PMESSAGE") def pmessage(self, message, *args, **kws): """ Performance Info/Message Level """ # Yes, logger takes its '*args' as 'args'. self._log(PMESSAGE_NUM, message, args, **kws) logging.Logger.pmessage = pmessage # Performance Debug... logging.addLevelName(PDEBUG_NUM, "PDEBUG") def pdebug(self, message, *args, **kws): """ Performance Debug Message Level """ # Yes, logger takes its '*args' as 'args'. self._log(PDEBUG_NUM, message, args, **kws) logging.Logger.pdebug = pdebug # Performance Verbose... logging.addLevelName(PVERBOSE_NUM, "PVERBOSE") def pverbose(self, message, *args, **kws): """ Performance Verbose Message Level """ # Yes, logger takes its '*args' as 'args'. self._log(PVERBOSE_NUM, message, args, **kws) logging.Logger.pverbose = pverbose # # Setup Logging. # def SetLogging(verbosity): """ Set logging file and level. """ global rootLogger logFormatter = logging.Formatter("[%(levelname)s] %(message)s") # Output formatting for message (i.e. date/time, thread, line #, message level, message). rootLogger = logging.getLogger() # Add a File to log too... fileHandler = logging.FileHandler('/tmp/log.txt') fileHandler.setFormatter(logFormatter) rootLogger.addHandler(fileHandler) # Put the message on the Console as well.. consoleHandler = logging.StreamHandler() consoleHandler.setFormatter(logFormatter) rootLogger.addHandler(consoleHandler) # Set what the logging level should be: level = logging.getLevelName(verbosity) # This turns the level into a number, which setLevel uses or logging.INFO for "verbosity" variable. rootLogger.setLevel(level) # This can be DEBUG, INFO, WARN, ERROR or CRITICAL to control verbosity. Custom Levels: PVERBOSE (51), PDEBUG (52), PWARNING (53) or PERROR (54) print 'Verbosity Level set to: %s' % level # # # verbosity = 'PERROR' SetLogging(verbosity) rootLogger.pverbose('Performance Verbose Level') rootLogger.pdebug('Performance Debug Level') rootLogger.pmessage('Performance Message Level') rootLogger.pwarning('Performance Warning Level') rootLogger.perror('Performance Error Level') From davea at davea.name Wed Feb 18 08:54:29 2015 From: davea at davea.name (Dave Angel) Date: Wed, 18 Feb 2015 08:54:29 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <65b209a0-33fc-4c1f-8af6-de3a4626623c@googlegroups.com> References: <54E34C68.6040700@davea.name> <65b209a0-33fc-4c1f-8af6-de3a4626623c@googlegroups.com> Message-ID: <54E49995.6040403@davea.name> On 02/18/2015 04:04 AM, janhein.vanderburg at gmail.com wrote: > On Tuesday, February 17, 2015 at 3:35:16 PM UTC+1, Chris Angelico wrote: >> Oh, incidentally: If you want a decent binary format for >> variable-sized integer, check out the MIDI spec. > > I did some time ago, thanks, and it is indeed a decent format. > I also looked at variations of that approach. > None of them beats Define "beats." You might mean beats in simplicity, or in elegance, or in clarity of code. But you probably mean in space efficiency, or "compression." But that's meaningless without a target distribution of values that you expect to encode. For example, if 99.9% of your values are going to be less than 255, then the most efficient byte encoding would be one that simply stores a value less than 255, and starts with an FF for larger values. It's almost irrelevant how it encodes those larger values. On the other hand, if most values are going to be in the 10,000 to 20,000 bit size range, and a few will be much smaller, and a few will be very much larger, then it would be very practical to start with a size field, say 16 bits, followed by the raw packed data. Naturally, the size field would need to have an escape value that indicates a larger field was needed. In fact, the size field could be encoded in a 7bits-per-byte manner, so it would encode an arbitrary sized number as well. > "my" concept of two counters that cooperatively specify field lengths and represented integer values. > >> I've tried to read through the original algorithm description, but I'm >> not entirely sure: How many payload bits per transmitted byte does it >> actually achieve? > > I don't think that payload bits per byte makes sense in this concept. > Correct. Presumably one means average payload bits per byte. First one would have to define what the "standard" unencoded variable length integer format was. Then one could call that size the payload size. Then, in order to compute an average, one would have to specify an expected, or target distribution of values. One then compares and averages the payload size for each typical value with the encoded size. -- DaveA From rosuav at gmail.com Wed Feb 18 09:16:00 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 01:16:00 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E49995.6040403@davea.name> References: <54E34C68.6040700@davea.name> <65b209a0-33fc-4c1f-8af6-de3a4626623c@googlegroups.com> <54E49995.6040403@davea.name> Message-ID: On Thu, Feb 19, 2015 at 12:54 AM, Dave Angel wrote: >>> I've tried to read through the original algorithm description, but I'm >>> not entirely sure: How many payload bits per transmitted byte does it >>> actually achieve? >> >> >> I don't think that payload bits per byte makes sense in this concept. >> > > Correct. Presumably one means average payload bits per byte. > > First one would have to define what the "standard" unencoded variable length > integer format was. Then one could call that size the payload size. Then, > in order to compute an average, one would have to specify an expected, or > target distribution of values. One then compares and averages the payload > size for each typical value with the encoded size. Average, or separately for different ranges. Alternatively, identify which ranges of numbers can be encoded with how many bytes. For instance, if you have a varlen length (in bytes) followed by that many packed bytes, you would have: 2: up to 1<<8-1 3: up to 1<<16-1 4: up to 1<<24-1 ... 128: up to 1<<1016-1 130: up to 1<<1024-1 131: up to 1<<1032-1 etc Using varlen directly gives: 1: up to 1<<7-1 2: up to 1<<14-1 3: up to 1<<21-1 etc So if your number is around about 1<<20, you know it's going to take 3 bytes to encode as varlen, or 4 to encode with the meta-length. Easy comparisons. I'm not sure how this algorithm compares, because it's extremely clever. ChrisA From tak.govoril at gmail.com Wed Feb 18 10:08:21 2015 From: tak.govoril at gmail.com (tak.govoril at gmail.com) Date: Wed, 18 Feb 2015 07:08:21 -0800 (PST) Subject: Assigning a function to sys.excepthook doesn't work in WSGI Message-ID: <717f2dd0-7065-4b55-a536-9a40a6c8b291@googlegroups.com> I want to generate an html page with http response status '200 OK' in case of an uncaught exception in my wsgi application, instead of web server's standard '500 Internal Server Error' response for such case. To do so, I've made the following sample code: import sys def application(environ, start_response): def exception(etype, evalue, trace): start_response('200 OK', [('Content-Type', 'text/plain')]) return ['Error'] sys.excepthook = exception 1/0 start_response('200 OK', [('Content-Type', 'text/plain')]) return ['OK'] But the function 'exception' is never called, and a standard '500 Internal Server Error' response is still generated by server in case of an uncaught exception. I looked through the documentation, but unable to find the answer. Are there any ways to handle uncaught by try..except exceptions under mod_wsgi? From nadeesh092 at gmail.com Wed Feb 18 10:30:51 2015 From: nadeesh092 at gmail.com (Nadeesh Dilanga) Date: Wed, 18 Feb 2015 21:00:51 +0530 Subject: about gsoc 2015 Message-ID: Hi, I'm a Computer Science undergradute student who like to participate in GSOC this year. Do you have any projects willing to publish for gsoc 2015. I am more familiar with Python. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hasanuzzaman.im at gmail.com Wed Feb 18 11:08:32 2015 From: hasanuzzaman.im at gmail.com (Md. Hasanuzzaman) Date: Wed, 18 Feb 2015 21:38:32 +0530 Subject: Fwd: how to collect old tweets from 03-20 Aug, 2014 for a particular location In-Reply-To: References: Message-ID: Hello, I am a PhD student. I am new to python and using TwitterSearch to collect tweets from 03-20 Aug for a particular location (geocode). I am trying to run the following piece of code. twitter = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET) results = twitter.cursor(twitter.search, q='............', max_id='501488707195924481', count=100, geocode='38.00,-91.00') for result in results: print (result['id_str']) However, it is not returning anything. Please help me in this regard. Can I collect old tweets without any keyword for a particular location? Thanks in advance. Regards, Mohammed -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Wed Feb 18 11:46:58 2015 From: davea at davea.name (Dave Angel) Date: Wed, 18 Feb 2015 11:46:58 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> Message-ID: <54E4C202.1050500@davea.name> On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: > encoding individual integers optimally without any assumptions about their values. > Contradiction in terms. -- DaveA From ian.g.kelly at gmail.com Wed Feb 18 11:51:14 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Feb 2015 09:51:14 -0700 Subject: Assigning a function to sys.excepthook doesn't work in WSGI In-Reply-To: <717f2dd0-7065-4b55-a536-9a40a6c8b291@googlegroups.com> References: <717f2dd0-7065-4b55-a536-9a40a6c8b291@googlegroups.com> Message-ID: On Wed, Feb 18, 2015 at 8:08 AM, wrote: > I want to generate an html page with http response status '200 OK' in case of an uncaught exception in my wsgi application, instead of web server's standard '500 Internal Server Error' response for such case. To do so, I've made the following sample code: > > import sys > > def application(environ, start_response): > def exception(etype, evalue, trace): > start_response('200 OK', [('Content-Type', 'text/plain')]) > return ['Error'] > > sys.excepthook = exception > > 1/0 > > start_response('200 OK', [('Content-Type', 'text/plain')]) > return ['OK'] > > But the function 'exception' is never called, and a standard '500 Internal Server Error' response is still generated by server in case of an uncaught exception. sys.excepthook is called just before the interpreter exits due to an exception. In a mod_wsgi environment, having the interpreter exit just because of an exception would be undesirable. I don't know exactly what it's doing under the hood, but I would assume that the exception never makes it to sys.excepthook because the gateway itself is catching the exception in order to generate the 500 response. > I looked through the documentation, but unable to find the answer. Are there any ways to handle uncaught by try..except exceptions under mod_wsgi? Here is what PEP 3333 has to say about error handling: https://www.python.org/dev/peps/pep-3333/#error-handling From kai.peters at gmail.com Wed Feb 18 11:57:38 2015 From: kai.peters at gmail.com (kai.peters at gmail.com) Date: Wed, 18 Feb 2015 08:57:38 -0800 (PST) Subject: Noob Parsing question In-Reply-To: References: Message-ID: > >> > Given > >> > > >> > data = '{[][]}' > >> > > >> > How can I efficiently get dictionaries for each of the data blocks framed by <> ? > >> > > >> > Thanks for any help > >> > >> The question here is: What _can't_ happen? For instance, what happens > >> if Fred's name contains a greater-than symbol, or a caret? > >> > >> If those absolutely cannot happen, your parser can be fairly > >> straight-forward. Just put together some basic splitting (maybe a > >> regex), and then split on the caret inside that. Otherwise, you may > >> need a more stateful parser. > >> > >> ChrisA > > > > The data string is guaranteed to be clean - no such irregularities occur. > > Okay! > > (Side point: You've stripped off all citations, here, so it's not > clear who said what. My shorthand signature isn't as useful as the > full line identifying date, time, and person. It's polite to keep > those lines, at least for the first level of quoting.) > > What you want can be done with a regular expression. (Yes, yes, I > know; now you have two problems.) > > >>> data = '{[][]}' > >>> re.findall("<.*?>",data) > ['', '', '', ''] > > >From there, you can crack open the different pieces: > > >>> for piece in re.findall("<.*?>",data): > ... d = {} > ... for elem in piece[1:-2].split("^"): > ... key, value = elem.split("=",1) > ... d[key] = value > ... print(d) > ... > {'c': '45.22', 'b': 'Fred', 'a': '14'} > {'b': 'Joe', 'a': '22'} > {'c': '3.20', 'a': '17'} > {'b': 'Soup', 'a': '72'} > > If you need some of those to be integers or floats, you'll need to do > some post-processing on it, but this guarantees that you get the data > out reliably. It depends on not having any of the special characters > "=^<>" inside the elements, but other than that, it should be safe. > > ChrisA Thanks for your help - much appreciated! KP From breamoreboy at yahoo.co.uk Wed Feb 18 12:00:10 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 18 Feb 2015 17:00:10 +0000 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E4C202.1050500@davea.name> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <54E4C202.1050500@davea.name> Message-ID: On 18/02/2015 16:46, Dave Angel wrote: > On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: > > >> encoding individual integers optimally without any assumptions about >> their values. >> > > Contradiction in terms. > I'm just pleased to see new blood coming through for my dream team, it's been a bit quiet recently. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From invalid at invalid.invalid Wed Feb 18 12:30:24 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 18 Feb 2015 17:30:24 +0000 (UTC) Subject: python implementation of a new integer encoding algorithm. References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> Message-ID: On 2015-02-18, Dave Angel wrote: > On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: > > >> encoding individual integers optimally without any assumptions about >> their values. > > Contradiction in terms. Ah, that depends on whether the encoding has to be lossless or not. For example: def encode_integer(i): return 0; Voila! Optimal encoding with no assumptions about intut values. It is, however, a bit lossy. -- Grant Edwards grant.b.edwards Yow! Did you move a lot of at KOREAN STEAK KNIVES this gmail.com trip, Dingy? From bryan0731 at gmail.com Wed Feb 18 12:43:40 2015 From: bryan0731 at gmail.com (Bryan Duarte) Date: Wed, 18 Feb 2015 10:43:40 -0700 Subject: Accessible tools Message-ID: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> Hello all, I have been posting to another group which directed me to this group. I am a blind software engineering student at Arizona State University. I am currently doing research and have decided to use Python as my developing language. I am in search of an accessible IDE or other tool set which will allow me to use the features with a screen reader. I have tried a number of tools but so far have been unsuccessful in finding one that allows me to use the tools and features of the IDE with my screen reader. I understand that I do not need an IDE to use Python, I am looking for an IDE to allow me to explore the classes and methods easily and quickly. I have tried the following tools. ? pyCharm ? iPy ? iPy Notebook ? Idle ? and Eclipse So far only Eclipse was accessible but I did still incounter some difficulties like the IDE freezing up when tool tips would pop up. Finally I am using a Mac mostly but can use Windows, or Linux if need be. Thank you and I appreciate any and all suggestions. From python at mrabarnett.plus.com Wed Feb 18 13:04:08 2015 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 18 Feb 2015 18:04:08 +0000 Subject: What the Pythons docs means by "container" ? In-Reply-To: <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> Message-ID: <54E4D418.5030109@mrabarnett.plus.com> On 2015-02-18 02:14, candide wrote: > Le mercredi 18 f?vrier 2015 01:50:16 UTC+1, Chris Angelico a ?crit : > >> So, what's a container? It's a thing that you put other objects >> into. > > I agree with this approach. The important point to consider here is > the last word in your definition : "into". There is the container and > there is the content (the objects into). The so-called built-in > containers (list, string, etc) are in conformance with this view. > Now, regarding a range object as a container disagrees completely > with the definition given in the PLR : there is no contents and hence > there is no container. For instance, range(10**6) doesn't hold any > kind of object, there are no reference to the int objects 0, 1, 2, > ... As the range's docstring explains, range returns a VIRTUAL > sequence. > It's a virtual, read-only container that contains integers. Try comparing range(10) with tuple(range(10)). Both contain integers. Both have a length. Both can be indexed. From breamoreboy at yahoo.co.uk Wed Feb 18 13:12:43 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 18 Feb 2015 18:12:43 +0000 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> Message-ID: On 18/02/2015 17:30, Grant Edwards wrote: > On 2015-02-18, Dave Angel wrote: >> On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: >> >> >>> encoding individual integers optimally without any assumptions about >>> their values. >> >> Contradiction in terms. > > Ah, that depends on whether the encoding has to be lossless or not. > > For example: > > def encode_integer(i): > return 0; > > Voila! Optimal encoding with no assumptions about intut values. > > It is, however, a bit lossy. > Is it this kind of coding that makes the PEP 393 FSR so poor? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From mimmocubiddu at yahoo.com Wed Feb 18 13:24:02 2015 From: mimmocubiddu at yahoo.com (Stefano Roma. Leo Fund. London.) Date: Wed, 18 Feb 2015 10:24:02 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From mimmocubiddu at yahoo.com Wed Feb 18 13:31:16 2015 From: mimmocubiddu at yahoo.com (Stefano Roma. Leo Fund. London.) Date: Wed, 18 Feb 2015 10:31:16 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From tjreedy at udel.edu Wed Feb 18 13:50:39 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 18 Feb 2015 13:50:39 -0500 Subject: PythonMagick on Windows DLL load failed In-Reply-To: References: Message-ID: On 2/16/2015 7:17 AM, julien levasseur wrote: > I am using Python 2.7 on Windows 8.1. > > Python 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] > on win32 I *strongly* suggest that you update to 2.7.9. > I installed ImageMagick from imagemagick.org > > > Then installed PythonMagick with pip from > http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonmagick When I have used that site (most recently last July), I have directly downloaded and unzipped files. I see that nearly all .zips have since been replaced with .whl wheels. > When I import PythonMagick it says: > > Traceback (most recent call last): > > File "magic.py", line 2, in > > import PythonMagick > > File "C:\Python27\lib\site-packages\PythonMagick\__init__.py", line > 1, in > > from . import _PythonMagick > > ImportError: DLL load failed: The application has failed to start > because its side-by-side configuration is incorrect. Please see the > application event log or use the command-line sxstrace.exe tool for more > detail. With 64 bit python, you must use 64 bit binaries. If so, search "side-by-side configuration is incorrect" (Google, etc). > Any clues on where do look? > Question on stackoverflow > http://stackoverflow.com/questions/28538973/pythonmagick-on-windows-dll-load-failed I see 'cgohlke', who I presume is the C. Gohlke who made the PythonMagick binary, has already given you his best guess. -- Terry Jan Reedy From ian.g.kelly at gmail.com Wed Feb 18 13:50:56 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Feb 2015 11:50:56 -0700 Subject: What the Pythons docs means by "container" ? In-Reply-To: <54E4D418.5030109@mrabarnett.plus.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> <54E4D418.5030109@mrabarnett.plus.com> Message-ID: On Wed, Feb 18, 2015 at 11:04 AM, MRAB wrote: > On 2015-02-18 02:14, candide wrote: >> >> Le mercredi 18 f?vrier 2015 01:50:16 UTC+1, Chris Angelico a ?crit : >> >>> So, what's a container? It's a thing that you put other objects >>> into. >> >> >> I agree with this approach. The important point to consider here is >> the last word in your definition : "into". There is the container and >> there is the content (the objects into). The so-called built-in >> containers (list, string, etc) are in conformance with this view. >> Now, regarding a range object as a container disagrees completely >> with the definition given in the PLR : there is no contents and hence >> there is no container. For instance, range(10**6) doesn't hold any >> kind of object, there are no reference to the int objects 0, 1, 2, >> ... As the range's docstring explains, range returns a VIRTUAL >> sequence. >> > It's a virtual, read-only container that contains integers. > > Try comparing range(10) with tuple(range(10)). Both contain integers. > Both have a length. Both can be indexed. Another way to look at range is as an optimization of the range from Python 2, which returned an actual list of integers. Most callers are only interested in iterating over the container and don't have to be concerned whether the result is actually a list or just something that looks enough like one for their purposes. From that point of view, a range object is still a container. From tjreedy at udel.edu Wed Feb 18 14:01:30 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 18 Feb 2015 14:01:30 -0500 Subject: Issues pip-installing pywin32 In-Reply-To: References: Message-ID: On 2/18/2015 7:13 AM, Chris Angelico wrote: > (Before I begin, I want to make it clear that I can still go poke > around on SourceForge and grab the appropriate installer [1], and that > does work. But if I'm going to tell someone else how to set up this > program, I'd much rather be able to recommend pip.) > > On a fresh Python 3.4 running on Windows 7, attempting to "pip install > pywin32" doesn't work: > > C:\Users\Rosuav>pip install pywin32 > Downloading/unpacking pywin32 > Could not find any downloads that satisfy the requirement pywin32 > Some externally hosted files were ignored (use --allow-external > pywin32 to allow). > Cleaning up... > No distributions at all found for pywin32 > Storing debug log for failure in C:\Users\Rosuav\pip\pip.log > > C:\Users\Rosuav>pip install --allow-external pywin32 > You must give at least one requirement to install (see "pip help install") > > > Okay, so the obvious command doesn't work, and gives a > not-particularly-helpful error message, but I know how to Google, I > can get past this. > > > C:\Users\Rosuav>pip install --allow-external pywin32 > --allow-unverified pywin32 pywin32 > Downloading/unpacking pywin32 > Could not find any downloads that satisfy the requirement pywin32 > Cleaning up... > No distributions at all found for pywin32 > Storing debug log for failure in C:\Users\Rosuav\pip\pip.log > > > Huh. The PyPI categorization seems to exclude Python 3.4 support I have pip-installed 3.4 packages just fine. > (despite versions for both 3.4 and 3.5 existing on the sourceforge > page). Let's try that with 2.7, just to see what happens. (I could > alternatively backlevel to 3.3, I suppose.) > > C:\Users\Rosuav>\python27\python -m pip --version > pip 1.5.6 from C:\python27\lib\site-packages (python 2.7) > > C:\Users\Rosuav>\python27\python -m pip install --allow-external > pywin32 --allow-unverified pywin32 pywin32 > Downloading/unpacking pywin32 > Could not find any downloads that satisfy the requirement pywin32 > Cleaning up... > No distributions at all found for pywin32 > Storing debug log for failure in C:\Users\Rosuav\pip\pip.log > > > The log contains these lines, which may be a clue as to what's going on... > > Could not fetch URL > https://sourceforge.net/projects/pywin32/files/pywin32/ (from > https://pypi.python.org/simple/pywin32/): connection error: hostname > 'sourceforge.net' doesn't match either of 'cloudfront.net', > '*.cloudfront.net' > Will skip URL > https://sourceforge.net/projects/pywin32/files/pywin32/ when looking > for download links for pywin32 > Could not find any downloads that satisfy the requirement pywin32 > > Does this mean there's an HTTPS configuration error at sourceforge? > And if that is indeed the problem, is there any way that I, as an end > user, can bypass the check? My alternative is using a non-SSL link, so > it's not like I'd actually have worse security. > > At this point, I am REALLY glad my Dad asked me to set this all up for > him, rather than just giving him some instructions and saying "go for > it". (And to think, all I want is a simple program to bounce some info > out from a Windows VM onto the Linux host. So much hassle for > something so simple.) > > [1] http://sourceforge.net/projects/pywin32/files/pywin32/ I would send your experience to Mark Hammond to let him know that pythonwin does not seem to be pip-installable. -- Terry Jan Reedy From tak.govoril at gmail.com Wed Feb 18 14:16:34 2015 From: tak.govoril at gmail.com (Alexander Sh) Date: Wed, 18 Feb 2015 11:16:34 -0800 (PST) Subject: Assigning a function to sys.excepthook doesn't work in WSGI In-Reply-To: References: <717f2dd0-7065-4b55-a536-9a40a6c8b291@googlegroups.com> Message-ID: <96de2b0d-7976-40b7-8d0f-221f208de3eb@googlegroups.com> On Wednesday, February 18, 2015 at 7:52:19 PM UTC+3, Ian wrote: > > sys.excepthook is called just before the interpreter exits due to an > exception. In a mod_wsgi environment, having the interpreter exit just > because of an exception would be undesirable. I don't know exactly > what it's doing under the hood, but I would assume that the exception > never makes it to sys.excepthook because the gateway itself is > catching the exception in order to generate the 500 response. > > > I looked through the documentation, but unable to find the answer. Are there any ways to handle uncaught by try..except exceptions under mod_wsgi? > > Here is what PEP 3333 has to say about error handling: > https://www.python.org/dev/peps/pep-3333/#error-handling Thank you for your reply, it clarifies everything. Actually, I've missed that PEP in my studies. From janhein.vanderburg at gmail.com Wed Feb 18 14:29:00 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 11:29:00 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> Message-ID: Op woensdag 18 februari 2015 10:36:37 UTC+1 schreef Chris Angelico: > I would actually look at it the other way: I'm aware of that, since you already warned me with "This is a fine forum to ask in. However, you may find that the advice you get isn't quite what you were asking for. In my case, ..." Again: no offense, but I agree to disagree. There will be no assumptions about the integers to be encoded. Jan-Hein. From ethan at stoneleaf.us Wed Feb 18 14:43:52 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 11:43:52 -0800 Subject: What the Pythons docs means by "container" ? In-Reply-To: <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> Message-ID: <54E4EB78.80402@stoneleaf.us> On 02/17/2015 06:14 PM, candide wrote: > Le mercredi 18 f?vrier 2015 01:50:16 UTC+1, Chris Angelico a ?crit : > >> >> So, what's a container? It's a thing that you put other objects into. > > I agree with this approach. The important point to consider here is the last word in your definition : "into". You are getting bogged down with implementation details; not even dict, list, nor tuple actually have the contained objects inside their memory space. What they have is a record of what objects they "contain", and methods to work with those objects. Whether a "contained" object exists before it is accessed is irrelevant, is an implementation detail, and is a level of optimization. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From janhein.vanderburg at gmail.com Wed Feb 18 14:48:38 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 11:48:38 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <85554c47-0957-4a5a-8ac7-513cd0f436ff@googlegroups.com> Message-ID: <4efc991a-947f-415f-9999-2f490aef0927@googlegroups.com> Op woensdag 18 februari 2015 11:33:18 UTC+1 schreef Laura Creighton: > Hi Jan. Hi Laura, thanks for your comments; let me explain my why: > Should you ever need an assembler programmer for > quick and dirty hacks for the PDP-11 line (11/20 and 11/05 preferred > as it is harder) I am still the woman for the job. Indeed, I spent > most of my 20s finding better and better ways to fit programs into > smaller and smaller memory footprints. I once wrote a bootstrap loader for a PDP-11, but I forget when and what model exactly, for it was some decades ago and not all that spectacular. > I perfectly understand the intellectual thrill of doing such things. > As puzzles go, it is about as cool a one as exists, and its all for > things that matter -- for real. > > However, in the matter of financial compensation and world recognition, > you have just laid a very large goose-egg. I don't want financial compensation and don't need world recognition, so I will accept the egg. > so all the techniques we learned for making our code concise -- let alone > the dirty tricks I specialised in -- were no longer relevant. If dirty tricks means shortcuts I personally wouldn't have excepted them even at that time of resource shortages. I hate shortcuts. > >From the mid 1980s onward I have been telling people 'your code is > ugly, please tighten it up by refactoring it and ' and > when I am their instructor they grumble and do it, and otherwise they > flip me the bird. In their eyes, it doesn't matter how the code > _looks_ as long as it does the job. Do not get intimidated and simply refuse to follow the lemmings. > what I am going for is not a 'death - by looking unfashionable' but > rather a demand that good code is clear to understand. Sorry that my code is all that confusing. I have had several comparable complaints from other contributors to this discussion. I will try to improve that part of my proposal, but don't hold your breath, for it may take some time. In fact I was initially asking for help with that code, but I ended up defending my credentials. > Your proposed encoding scheme (if it does as you say, I have not > analysed it) scores very, very high in the _cleverness_ department. > Enough that a lot of people, who aren't as clever as you do, have > no hope in hell of ever being clever enough to debug something that > uses it. Therefore, you will never see widespread adoption of your > scheme, no matter how brilliantly it does as you say, because we all > need things that are easier to debug more than we need better compression. I don't need widespread adaption outside my own applications, and I don't see any problem for a python programmer that uses an object that flawlessly encodes and decodes an arbitrary integer value. To me debugging is not synonymous to making things human readable or even translatable. So what's in it for other programmers then me myself and I?: the no brainer optimizing of resource requirements. Compression is different from optimal individual integer encoding, since the first needs a context of surrounding integers in a sequence of integer images. > So now you are sad. Not yet. > So then, now what? > > If you are still fired up with the desire to compress things, then > there is a huge, _very well paying_ market I want to introduce you > to. And this is _tech support for porn sites_. Porn sites make a > ton of money, indeed the numbers are scary. And here the idea of > 'I saved 2% of time/bandwidth/disk space/' really matters. You > can really save money for them, and it really matters to them. > Since I have never found sex 'dirty' and indeed consider it one > of the great joys in life, I have never found anything wrong with > working for porn sites. Frankly a have a first application in mind that is less sexy and won't be all that profitable. There is nothing wrong with porn indeed, but at our age it is just no longer on the top of one's mind every second is it? > And, hell, out of male porn Jimmy Wales made wikipedia. Haven't > we all benefitted? I really don't know what you are talking about here, and only interested in a reply if it seriously contributes to my personal objective to get "my" way of integer encoding implemented as proposed. > But, right now, you are, alas for you, about 45 years too late for the > ideas you are sprouting. I had similar ones about 30 years too late > and, well, they only worked for me for about 3-5 years. Sucks to be > you, friend -- you needed to be your grandfather, I fear. Thanks for making me feel so mature. Let's go back to the future. From janhein.vanderburg at gmail.com Wed Feb 18 14:52:21 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 11:52:21 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <54E34C68.6040700@davea.name> <65b209a0-33fc-4c1f-8af6-de3a4626623c@googlegroups.com> Message-ID: Op woensdag 18 februari 2015 14:55:07 UTC+1 schreef Dave Angel: > Define "beats." You might mean beats in simplicity, or in elegance, or > in clarity of code. But you probably mean in space efficiency, or > "compression." But that's meaningless without a target distribution of > values that you expect to encode. I do indeed mean space efficiency, not compression. And it is not meaningless and I don't do distribution tricks. Don't take this personally. From janhein.vanderburg at gmail.com Wed Feb 18 14:55:28 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Wed, 18 Feb 2015 11:55:28 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> Message-ID: <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel: > On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: > > > > encoding individual integers optimally without any assumptions about their values. > > > > Contradiction in terms. > > -- > DaveA Not. Jan-Hein. From tjreedy at udel.edu Wed Feb 18 15:04:12 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 18 Feb 2015 15:04:12 -0500 Subject: What the Pythons docs means by "container" ? In-Reply-To: <54E4EB78.80402@stoneleaf.us> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> <54E4EB78.80402@stoneleaf.us> Message-ID: On 2/18/2015 2:43 PM, Ethan Furman wrote: > Whether a "contained" object exists before it is accessed is irrelevant, is an implementation detail, and is a level of > optimization. Is in not irrelevant in that virtual collections can be infinite. -- Terry Jan Reedy From ben+python at benfinney.id.au Wed Feb 18 15:14:19 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 07:14:19 +1100 Subject: sqlite3 and dates References: Message-ID: <853863asf8.fsf@benfinney.id.au> Johannes Bauer writes: > On 18.02.2015 08:05, Chris Angelico wrote: > > > But if you need more facilities than SQLite3 can offer, maybe it's > > time to move up to a full database server, instead of local files. > > Switching to PostgreSQL will give you all those kinds of features, > > plus a lot of other things that I would have thought pretty basic - > > like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't > > support that. > > I see you're running a lawnmower. Maybe you should switch to a combine > harvester. That'll get you extra features like a reciprocating knife > cutter bar. I was quite surprised that regular lawnmowers don't support > those. Chris has pointed out one flaw in this analogy; I'll address another. A feature like ?ALTER TABLE? is not equivalent to a ?reciprocating knife cutter bar?. I'm in agreement that it is a pretty basic SQL feature, and it doesn't appear to conflict with the narrow focus that we all agree is appropriate for SQLite. So you're mocking such an expectation as though it's expecting something wildly niche. I think you're propping up a straw man there; the expectation is quite simple and its absence from SQLite is astonishing. Your attempted mockery does not, IMO, hit home. -- \ ?When we call others dogmatic, what we really object to is | `\ their holding dogmas that are different from our own.? ?Charles | _o__) Issawi | Ben Finney From ian.g.kelly at gmail.com Wed Feb 18 15:15:23 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Feb 2015 13:15:23 -0700 Subject: Logging with Custom Levels not working In-Reply-To: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> References: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> Message-ID: On Wed, Feb 18, 2015 at 6:49 AM, Didymus wrote: > def perror(self, message, *args, **kws): > """ Performance Error Message Level """ > # Yes, logger takes its '*args' as 'args'. > self._log(PERROR_NUM, message, args, **kws) > > logging.Logger.perror = perror I think you need to call self.log, not self._log. The _log method appears to assume that the level check has already been performed. You really shouldn't be calling it directly anyway, as the leading _ is an indication that the method is not part of the public API. From JKZMM at clemson.edu Wed Feb 18 15:56:03 2015 From: JKZMM at clemson.edu (James Zimmerman) Date: Wed, 18 Feb 2015 20:56:03 +0000 Subject: can python handle CHIME .spt files? Message-ID: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> I have a lot of old files that were written in Chime, then made compatible with Jmol. Now there are problems with that. Will Python give me a way to be able to show the .spt files that Chime could read? I don't want to start learning Python unless I know there is a way. Thank you very much. James Zimmerman Emeritus Professor of Biochemistry -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Feb 18 16:53:22 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 13:53:22 -0800 Subject: can python handle CHIME .spt files? In-Reply-To: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> Message-ID: <54E509D2.7040200@stoneleaf.us> On 02/18/2015 12:56 PM, James Zimmerman wrote: > > I have a lot of old files that were written in Chime, then made compatible with Jmol. > Now there are problems with that. Will Python give me a way to be able to show the > .spt files that Chime could read? I don?t want to start learning Python unless I > know there is a way. This isn't my field, but I found: http://higheredbcs.wiley.com/legacy/college/voet/0470129301/bioinfo_stu/sect4.html Some of the nice things about Python is it's fairly easy to learn and has good community support. Good luck! -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From marko at pacujo.net Wed Feb 18 16:54:05 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 18 Feb 2015 23:54:05 +0200 Subject: python implementation of a new integer encoding algorithm. References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> Message-ID: <87fva2ophe.fsf@elektro.pacujo.net> janhein.vanderburg at gmail.com: > Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel: >> On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: >> > encoding individual integers optimally without any assumptions >> > about their values. >> >> Contradiction in terms. > > Not. Out of curiosity, could you give me an example of an integer, not assuming anything about its value. I mean, any integer you could mention would be very close to zero compared with virtually all other integers. Marko From python at discontinuity.net Wed Feb 18 16:55:03 2015 From: python at discontinuity.net (Davin Potts) Date: Wed, 18 Feb 2015 15:55:03 -0600 Subject: can python handle CHIME .spt files? In-Reply-To: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> Message-ID: <386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> Hi James ? I?m not entirely sure I understand what you?re wanting to do, but my ears perked up when you mentioned Jmol and Chime so I?ll try to offer some pointers: * If you want to assess which .spt files are failing when trying to read them with Jmol, then yes, Python can be used to automate the feeding of files to Jmol and can capture whether the loading is successful or not. * If you want a way to parse problematic .spt files that can no longer be read in using Jmol or MDL Chime, it is possible to create an .spt parser in Python capable of reading your files though my quick searches for an existing .spt parser library in Python came up empty. If this is your goal (to read those problematic .spt files), have you considered using OpenBabel to read them and convert them to another mainstream chemical structure format (SDF, MOL, etc.), cleaned up and ready for reading by any of a variety of other tools? You may already be aware but just in case it isn?t evident, Python itself does not ship with capabilities for reading/writing chemical structure formats but there are lots of chemistry tools built on top of Python that do. You may well find it most productive to pick up one of those tools in lieu of Python itself. Hope this helps, Davin > On Feb 18, 2015, at 2:56 PM, James Zimmerman wrote: > > I have a lot of old files that were written in Chime, then made compatible with Jmol. Now there are problems with that. Will Python give me a way to be able to show the .spt files that Chime could read? I don?t want to start learning Python unless I know there is a way. > > Thank you very much. > > James Zimmerman > Emeritus Professor of Biochemistry > -- > https://mail.python.org/mailman/listinfo/python-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Feb 18 17:06:50 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 14:06:50 -0800 Subject: can python handle CHIME .spt files? In-Reply-To: <386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU> <386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> Message-ID: <54E50CFA.2090307@stoneleaf.us> On 02/18/2015 01:55 PM, Davin Potts wrote: > * If you want a way to parse problematic .spt files that can no longer be read in using Jmol or MDL Chime, it is > possible to create an .spt parser in Python capable of reading your files though my quick searches for an existing .spt > parser library in Python came up empty. It wasn't easy to find, but the previous link I posted did lead to this: http://www.pymol.org/ -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From marko at pacujo.net Wed Feb 18 17:08:26 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 19 Feb 2015 00:08:26 +0200 Subject: python implementation of a new integer encoding algorithm. References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <87fva2ophe.fsf@elektro.pacujo.net> Message-ID: <87bnkqooth.fsf@elektro.pacujo.net> Marko Rauhamaa : > Out of curiosity, could you give me an example of an integer, not > assuming anything about its value. > > I mean, any integer you could mention would be very close to zero > compared with virtually all other integers. And I don't mean to be snide, either. I'm just saying that your "optimal encoding" necessarily depends on the expected distribution of the integers. There is no such thing as an even distribution of integers. Marko From rurpy at yahoo.com Wed Feb 18 17:13:58 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 14:13:58 -0800 (PST) Subject: sqlite3 and dates In-Reply-To: References: Message-ID: <9799edb6-d01d-49c2-82a4-3d70d9ca2427@googlegroups.com> On 02/18/2015 01:14 PM, Ben Finney wrote: > Johannes Bauer writes: >> On 18.02.2015 08:05, Chris Angelico wrote: >> >>> But if you need more facilities than SQLite3 can offer, maybe it's >>> time to move up to a full database server, instead of local files. >>> Switching to PostgreSQL will give you all those kinds of features, >>> plus a lot of other things that I would have thought pretty basic - >>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >>> support that. >> >> I see you're running a lawnmower. Maybe you should switch to a combine >> harvester. That'll get you extra features like a reciprocating knife >> cutter bar. I was quite surprised that regular lawnmowers don't support >> those. > > Chris has pointed out one flaw in this analogy; I'll address another. > > A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife > cutter bar". I'm in agreement that it is a pretty basic SQL feature, and > it doesn't appear to conflict with the narrow focus that we all agree is > appropriate for SQLite. No, you and Chris are way off base and Johannes is correct. He was pointing out that there are many applications that can benefit from a database and a full-blown, bells and whistles solution like Postgresql is often overkill in that (very common) case. His analogy is quite apt and I wish I'd thought of it. > So you're mocking such an expectation as though it's expecting something > wildly niche. I think you're propping up a straw man there; the > expectation is quite simple and its absence from SQLite is astonishing. > Your attempted mockery does not, IMO, hit home. It has already been pointed out that Sqlite *does* have ALTER TABLE so you are either deliberately propagating false information or not paying attention. The only thing basic that a (claimed) relational database needs to support is the basic relational operations of which ALTER TABLE is not one, And if one insists on ALTER TALE the minimal requirement there is that it support RENAME. You and Chris are confusing convenience with necessity. The success and wide adoption of Sqlite is pretty clear evidence that there are many applications that can benefit from its use. Sqllite gets used because of a very important feature that Postgresql is missing -- the ability to embed a database in an application without requiring the installation and maintenance of an entire separate client-server infrastructure. If you are tempted to trivialize that feature you might want to scan the Postgresql mail list where that feature is regularly requested for Postgresql. From rurpy at yahoo.com Wed Feb 18 17:17:07 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 14:17:07 -0800 (PST) Subject: sqlite3 and dates In-Reply-To: References: Message-ID: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> On 02/18/2015 04:21 AM, Chris Angelico wrote: > On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer wrote: >> On 18.02.2015 08:05, Chris Angelico wrote: >> >>> But if you need more facilities than SQLite3 can offer, maybe it's >>> time to move up to a full database server, instead of local files. >>> Switching to PostgreSQL will give you all those kinds of features, >>> plus a lot of other things that I would have thought pretty basic - >>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >>> support that. >> >> I see you're running a lawnmower. Maybe you should switch to a combine >> harvester. That'll get you extra features like a reciprocating knife >> cutter bar. I was quite surprised that regular lawnmowers don't support >> those. > > SQLite3 is fine for something that's basically just a more structured > version of a flat file. You assume that nobody but you has the file > open, and you manipulate it just the same as if it were a big fat blob > of JSON, but thanks to SQLite, you don't have to rewrite the whole > file every time you make a small change. That's fine. That's bullshit. Sqlite offers a lot more than that including a SQL interface, transactions, referential integrity, constraints indexes, triggers and other general relational database features. That you would equate that to a JSON blob would indicate either a profound ignorance about Sqlite or (more likely) a need to defend your preference with complete disregard of fact. > But it's the > wrong tool for any job involving multiple users over a network, and > quite probably the wrong tool for a lot of other jobs too. Nobody disputes that nor was that the point. The point was that there are many applications that can benefit from use of a database that are NOT distributed multi-user, muilti-tier applications. For many of that very large class of applications Sqlite is a good (and preferable to Postgresql) solution. > It's the > smallest-end piece of software that can truly be called a database. I > would consider it to be the wrong database for serious accounting > work, and that's based on the ranting of a majorly-annoyed accountant > who had to deal with issues in professional systems that had made > similar choices in back-end selection. I consider the program I use for my personal accounting program to be for very serious use since errors could have very grave consequences for me. But a multi-user client-server database is emphatically not needed by it. And I'm sure you're aware that "not for serious use" is a common way that C and Java programmers dismiss Python? So maybe you should try a little harder to come up with real arguments and not rely on cheap and meaningless labels. > You're welcome to disagree, but since PostgreSQL doesn't cost any > money and (on Linux at least; can't speak for other platforms) doesn't > take significant effort to set up, I will continue to recommend it. Postgresql costs a *lot* more -- in setup and on-going maintenance. Not all (or even most) costs are the initial monetary purchase expense. Its an unmoderated newsgroup so feel free to recommend what you want. But counter opinions should be aired so that others a can judge for themselves whether your recommendations are based on facts or on your personal preferences. From davea at davea.name Wed Feb 18 17:19:48 2015 From: davea at davea.name (Dave Angel) Date: Wed, 18 Feb 2015 17:19:48 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> Message-ID: <54E51004.1000104@davea.name> On 02/18/2015 02:55 PM, janhein.vanderburg at gmail.com wrote: > Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel: >> On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: >> >> >>> encoding individual integers optimally without any assumptions about their values. >>> >> >> Contradiction in terms. >> >> -- >> DaveA > > Not. > Jan-Hein. > Then you had better define your new word "optimal" to us. I decided to try your algorithm for all the values between 0 and 999999. One million values, and the 7bit encoding[1] beat yours for 950081 of them. The rest were tied. Yours never was shorter. For a uniform distribution of those particular values, the average number of bytes used by yours was 3.933568 bytes, and by 7bit encoding was 2.983487 For the second and third million, yours are all 4 bytes, while 7bit uses 3. Beyond 2097152, 7bit uses 4 bytes, same as you. Between 16 and 17 million, you average 4.156865, while 7bit is a constant 4.0. After that, I started spot-checking. I went up to 100 billion, and for none of those I tried did your algorithm take fewer bytes than 7bit. So how is yours optimal? Over what range of values? I'm not necessarily doubting it, just challenging you to provide a data sample that actually shows it. And of course, I'm not claiming that 7bit is in any way optimal. You cannot define optimal without first defining the distribution. [1] by 7bit, I'm referring to the one apparently used in MIDI encoding, where 7 bits of each byte hold the value, and the 8th bit is zero, except for the last byte, where the 8th bit is one. So 3 bytes can encode 21 bits, or up to 2**21 - 1. -- DaveA From rosuav at gmail.com Wed Feb 18 17:37:49 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 09:37:49 +1100 Subject: sqlite3 and dates In-Reply-To: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> Message-ID: On Thu, Feb 19, 2015 at 9:17 AM, wrote: >> SQLite3 is fine for something that's basically just a more structured >> version of a flat file. You assume that nobody but you has the file >> open, and you manipulate it just the same as if it were a big fat blob >> of JSON, but thanks to SQLite, you don't have to rewrite the whole >> file every time you make a small change. That's fine. > > That's bullshit. Sqlite offers a lot more than that including > a SQL interface, transactions, referential integrity, constraints > indexes, triggers and other general relational database features. > > That you would equate that to a JSON blob would indicate either > a profound ignorance about Sqlite or (more likely) a need to > defend your preference with complete disregard of fact. I didn't equate them. I said that SQLite3 is great if you look on it as an upgrade over a JSON blob. Of course it offers more features than that, and you don't need to swear at me to make your point. But SQLite3 is *not* great if you look on it as a database engine comparable with DB2, PostgreSQL, and even MySQL. ChrisA From ethan at stoneleaf.us Wed Feb 18 17:52:36 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 14:52:36 -0800 Subject: sqlite3 and dates In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> Message-ID: <54E517B4.4000409@stoneleaf.us> On Thu, Feb 19, 2015 at 9:17 AM, rurpy wrote: > That you would equate that to a JSON blob [...] Chris wrote: > I didn't equate them. >> Chris wrote earlier: >>> and you manipulate it just the same as if it were a big fat blob >>> of JSON That sure sounds like equating. Chris also wrote: > But SQLite3 is *not* great if you look on it as a database engine > comparable with DB2, PostgreSQL, and even MySQL. Sure, the LITE in SQLite means you don't get some things. There is still a huge amount of software that doesn't need concurrency and can benefit from it. Having installed Postgres I can say there is definitely a cost to install it, use it, maintain it, etc... especially if you aren't steeped in it and have to look things up every time you have to make a change (how do I add a user again?). I think the general advice should be: if you are writing a single-user application that happens to need SQL services, check out SQLite; if you are writing a multi-user or concurrent SQL application, check out Postgres. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rosuav at gmail.com Wed Feb 18 17:54:53 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 09:54:53 +1100 Subject: Issues pip-installing pywin32 In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 6:01 AM, Terry Reedy wrote: > On 2/18/2015 7:13 AM, Chris Angelico wrote: >> Huh. The PyPI categorization seems to exclude Python 3.4 support > > > I have pip-installed 3.4 packages just fine. > > >> (despite versions for both 3.4 and 3.5 existing on the sourceforge >> page). Let's try that with 2.7, just to see what happens. (I could >> alternatively backlevel to 3.3, I suppose.) I know pip works with 3.4 generally; I was talking specifically about pywin32 - if you look it up on PyPI, it doesn't mention 3.4 support. But it does mention 2.7, which is why I then tried that. > I would send your experience to Mark Hammond to let him know that pythonwin > does not seem to be pip-installable. Thanks, will do. ChrisA From invalid at invalid.invalid Wed Feb 18 17:58:47 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 18 Feb 2015 22:58:47 +0000 (UTC) Subject: python implementation of a new integer encoding algorithm. References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <87fva2ophe.fsf@elektro.pacujo.net> Message-ID: On 2015-02-18, Marko Rauhamaa wrote: > janhein.vanderburg at gmail.com: > >> Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel: >>> On 02/18/2015 03:59 AM, janhein.vanderburg at gmail.com wrote: >>> > encoding individual integers optimally without any assumptions >>> > about their values. >>> >>> Contradiction in terms. >> >> Not. > > Out of curiosity, could you give me an example of an integer, not > assuming anything about its value. > > I mean, any integer you could mention would be very close to zero > compared with virtually all other integers. Thus demonstrating the fitness of my previously posted "lossy integer encoding" algorithm. -- Grant Edwards grant.b.edwards Yow! Hmmm ... an arrogant at bouquet with a subtle gmail.com suggestion of POLYVINYL CHLORIDE ... From steve+comp.lang.python at pearwood.info Wed Feb 18 18:03:47 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 19 Feb 2015 10:03:47 +1100 Subject: What the Pythons docs means by "container" ? References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> <54E4EB78.80402@stoneleaf.us> Message-ID: <54e51a54$0$12994$c3e8da3$5496439d@news.astraweb.com> Terry Reedy wrote: > On 2/18/2015 2:43 PM, Ethan Furman wrote: > >> Whether a "contained" object exists before it is accessed is irrelevant, >> is an implementation detail, and is a level of optimization. > > Is in not irrelevant in that virtual collections can be infinite. Hmmm. I'm not sure I believe that. Can you enumerate all of the items and see if there actually is an infinite number of them? I'll wait... *wink* But seriously... Ethan's comment shouldn't be interpreted as meaning that virtual containers are exactly the same as concrete ones. Ethan's comment should be understand in terms that whether something is a container or not is *not* connected to whether it is a concrete data structure like a list or an array, or a virtual/lazy/on-demand data structure like (x)range. In Python ABC terms, I believe that the, or at least a, defining characteristic of containers is that they implement the "in" operator. If there is some value X for which `X in OBJ` succeeds, then OBJ may be a container. -- Steven From steve+comp.lang.python at pearwood.info Wed Feb 18 18:07:21 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 19 Feb 2015 10:07:21 +1100 Subject: sqlite3 and dates References: <9799edb6-d01d-49c2-82a4-3d70d9ca2427@googlegroups.com> Message-ID: <54e51b29$0$12994$c3e8da3$5496439d@news.astraweb.com> rurpy at yahoo.com wrote: > On 02/18/2015 01:14 PM, Ben Finney wrote: >> Johannes Bauer writes: >>> On 18.02.2015 08:05, Chris Angelico wrote: >>> >>>> But if you need more facilities than SQLite3 can offer, maybe it's >>>> time to move up to a full database server, instead of local files. >>>> Switching to PostgreSQL will give you all those kinds of features, >>>> plus a lot of other things that I would have thought pretty basic - >>>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >>>> support that. >>> >>> I see you're running a lawnmower. Maybe you should switch to a combine >>> harvester. That'll get you extra features like a reciprocating knife >>> cutter bar. I was quite surprised that regular lawnmowers don't support >>> those. >> >> Chris has pointed out one flaw in this analogy; I'll address another. >> >> A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife >> cutter bar". I'm in agreement that it is a pretty basic SQL feature, and >> it doesn't appear to conflict with the narrow focus that we all agree is >> appropriate for SQLite. > > No, you and Chris are way off base and Johannes is correct. > He was pointing out that there are many applications that can > benefit from a database and a full-blown, bells and whistles > solution like Postgresql is often overkill in that (very common) > case. His analogy is quite apt and I wish I'd thought of it. I'm not seeing that at all. Chris explicitly proceeded his comments with the condition "if you need more facilities than SQLite3 can offer". Johannes' analogy ignores that and consequently mocks the very idea that anyone might need more than a regular lawmower -- even a farmer with a thousand acres of wheat to be harvested. Johannes' subsequent posts are more nuanced about Sqlite filling a niche and not being suitable for everything, but the analogy you're supporting doesn't. It's an amusing quip, quite funny, but like most quips, lacks nuance and misrepresents Chris' original position. Had Chris said, "SQlite? Pah, don't use that, Postgresql is a much better solution!", the combine harvester analogy would have been much more fair. But he didn't, so it isn't. But fair or not, it has inspired good discussion, so there is that in it's favour. -- Steven From memilanuk at gmail.com Wed Feb 18 18:32:36 2015 From: memilanuk at gmail.com (memilanuk) Date: Wed, 18 Feb 2015 15:32:36 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <54E517B4.4000409@stoneleaf.us> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: On 02/18/2015 02:52 PM, Ethan Furman wrote: > Chris also wrote: >> But SQLite3 is *not* great if you look on it as a database engine >> comparable with DB2, PostgreSQL, and even MySQL. > > Sure, the LITE in SQLite means you don't get some things. There is still a huge amount of software that doesn't need > concurrency and can benefit from it. > > Having installed Postgres I can say there is definitely a cost to install it, use it, maintain it, etc... especially if > you aren't steeped in it and have to look things up every time you have to make a change (how do I add a user again?). > > I think the general advice should be: if you are writing a single-user application that happens to need SQL services, > check out SQLite; if you are writing a multi-user or concurrent SQL application, check out Postgres. Okay... this might be a question with a blindingly obvious answer, but I haven't seen any recommendations otherwise so I'll ask anyway ;) Is there anything *good* that sits in between the two extremes of SQLite and PostgreSQL? I've tinkered with MySQL years ago (in conjunction with PHP) and was a little unhappy with some of the things it either didn't implement fully (foreign keys) or silently ignored (check constraints). PostgreSQL, to me, is orders of magnitude harder to set up and maintain, though. And then there is SQLite, which does 99% of what I want it to do other than network use. I see other DB names such as DB2, Oracle, MS SQL Server, etc. out there but the only other 'free' one seems to be Firebird? Is that really the only other contender? Is there nothing that amounts to a 'PostgreSQLite'? -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From ben+python at benfinney.id.au Wed Feb 18 19:03:53 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 11:03:53 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: <85lhjuahsm.fsf@benfinney.id.au> memilanuk writes: > Okay... this might be a question with a blindingly obvious answer, but > I haven't seen any recommendations otherwise so I'll ask anyway ;) > > Is there anything *good* that sits in between the two extremes of > SQLite and PostgreSQL? What do you need a RDBMS to do, and what do you not need? The answers to those questions vary hugely between different people (and most people probably don't think too deeply about them). They will determine what ?good? means for your case. > Is there nothing that amounts to a 'PostgreSQLite'? PostgreSQL itself fits that mould quite well; it is quite capable of serving a small footprint while still offering full concurrency. I don't know of a free-software concurrent RDBMS which can be considered lighter than that. (No, MySQL doesn't count; its concurrency is *unreliable* and it commonly loses data silently. Don't use MySQL.) But perhaps you don't need concurrency? Only you can tell us. -- \ ?It's up to the masses to distribute [music] however they want | `\ ? The laws don't matter at that point. People sharing music in | _o__) their bedrooms is the new radio.? ?Neil Young, 2008-05-06 | Ben Finney From ben+python at benfinney.id.au Wed Feb 18 19:05:52 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 11:05:52 +1100 Subject: What the Pythons docs means by "container" ? References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> <54E4EB78.80402@stoneleaf.us> <54e51a54$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85h9uiahpb.fsf@benfinney.id.au> Steven D'Aprano writes: > Terry Reedy wrote: > > > [?] virtual collections can be infinite. > > Hmmm. I'm not sure I believe that. Can you enumerate all of the items > and see if there actually is an infinite number of them? I'll wait... A mathematician should know better: you don't need to enumerate to demonstrate a collection is infinite. Provided, of course, that the collection is *virtual* ? so virtual that it's not actually implemented on hardware :-) -- \ ?If you have the facts on your side, pound the facts. If you | `\ have the law on your side, pound the law. If you have neither | _o__) on your side, pound the table.? ?anonymous | Ben Finney From marfig at gmail.com Wed Feb 18 19:08:19 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 01:08:19 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: On Wed, 18 Feb 2015 15:32:36 -0800, memilanuk wrote: > >Is there anything *good* that sits in between the two extremes of SQLite >and PostgreSQL? > >I've tinkered with MySQL years ago (in conjunction with PHP) and was a >little unhappy with some of the things MariaDB is backwards compatible with MySQL and may answer some of the shortcomings. It's a much stronger RDBM in my opinion than MySQL and offers enterprise level features at cost 0. > PostgreSQL, to me, is orders of magnitude harder to set up and > maintain, though. PostgreSQL grows on you. It takes time to mature into a love relationship, like a complicated girlfriend (or boyfriend, whatever floats your boat). But once that relationship grows, you will want to marry with it, have little postgre kids and grow old with it. No other database stands a chance from that moment on. It's just too powerful and too feature rich, to ignore. Only Oracle stands a chance against it, in my humble opinion. And postgre isn't really that hard to setup and maintain. In fact, maintenance can be largely scriptable and 'croned' because the postgre server is so damn stable. Once you familiarize yourself with the process, you just realize it was easy all the time after all. I usually think of my relationship with postgre as similar to what I experienced with Git. At first I was just dumbstruck by the whole thing and my first reaction was to ignore it and just do version control as I knew with the tools I knew. But once my brain clicked into 'Git mode' and I realized its philosophy and its processes, I immediately recognized the benefits and understood why everyone was telling me Git was easy to use and highly useful. >then there is SQLite, which does 99% of what I want it to do other than >network use. SQLite misses some important features that makes it better suited as a simple datastore, not much unlike shelve. And network use is not one of them, since you can actually implement concurrent sqlite access by coding an intermediate layer. Assuming of course we are talking about a small number of concurrent users. Stored procedures is perhaps the most obvious missing feature. Contrary to an opinion I read on the thread that spawned this one, you really should thrive to put the business logic into the database as this permits great simplification of your code and much better adaptability to new requirements. SQLite IS a database. And wants to be used as a database. So despite agreeing SPs would increase SQLite footprint, it's undeniable they could be put to good use. Admittedly these too can be implemented through an intermediate layer. But are much more complex to code. Parameterized queries is just a pet peeve of mine that I wish to include here. SQLite misses it and I miss the fact SQLite misses it. The less SQL one needs to write in their code, the happier one should be. From rosuav at gmail.com Wed Feb 18 19:33:06 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 11:33:06 +1100 Subject: What the Pythons docs means by "container" ? In-Reply-To: <85h9uiahpb.fsf@benfinney.id.au> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <472c49ae-c2ce-4eda-b922-f16613722e31@googlegroups.com> <1d87da9c-ef64-4bd2-9179-e2aad908a9d0@googlegroups.com> <54E4EB78.80402@stoneleaf.us> <54e51a54$0$12994$c3e8da3$5496439d@news.astraweb.com> <85h9uiahpb.fsf@benfinney.id.au> Message-ID: On Thu, Feb 19, 2015 at 11:05 AM, Ben Finney wrote: >> Hmmm. I'm not sure I believe that. Can you enumerate all of the items >> and see if there actually is an infinite number of them? I'll wait... > > A mathematician should know better: you don't need to enumerate to > demonstrate a collection is infinite. But you may have to demonstrate a 1:1 correspondence with counting numbers, to prove that it's countably infinite. And that's pretty much what the enumerate() function does, so that's enumeration.... right? Bizarre pseudo-logic for the win. ChrisA From rosuav at gmail.com Wed Feb 18 19:42:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 11:42:58 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: On Thu, Feb 19, 2015 at 11:08 AM, Mario Figueiredo wrote: > I usually think of my relationship with postgre as similar to what I > experienced with Git. At first I was just dumbstruck by the whole > thing and my first reaction was to ignore it and just do version > control as I knew with the tools I knew. But once my brain clicked > into 'Git mode' and I realized its philosophy and its processes, I > immediately recognized the benefits and understood why everyone was > telling me Git was easy to use and highly useful. (Side point: If you're going to treat PostgreSQL the way you'd treat a girlfriend/boyfriend, you should probably be careful of how you address him. "Postgres" or "PostgreSQL", but not usually "Postgre".) This is a quite apt analogy. You have to get your head around some fundamentals, but once you do, life becomes amazing. >>then there is SQLite, which does 99% of what I want it to do other than >>network use. > > SQLite misses some important features that makes it better suited as a > simple datastore, not much unlike shelve. And network use is not one > of them, since you can actually implement concurrent sqlite access by > coding an intermediate layer. Assuming of course we are talking about > a small number of concurrent users. This is what I was saying: it's fine for purposes like Firefox's bookmarks and settings and such (which I think was what it was originally developed for?). Not so fine over a network. Adding an intermediate layer is a lot more effort than you might think. By the time you've gone there, you should be looking at PostgreSQL anyway. I tried to bolt networking support onto a couple of different databasing systems, back in the 90s, and it was faintly ridiculous... I mean, it worked, but if I'd had today's Postgres, I would never have done anything of the sort. ChrisA From steve+comp.lang.python at pearwood.info Wed Feb 18 21:13:08 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 19 Feb 2015 13:13:08 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: <54e546b5$0$11119$c3e8da3@news.astraweb.com> Chris Angelico wrote: >> SQLite misses some important features that makes it better suited as a >> simple datastore, not much unlike shelve. And network use is not one >> of them, since you can actually implement concurrent sqlite access by >> coding an intermediate layer. Assuming of course we are talking about >> a small number of concurrent users. > > This is what I was saying: it's fine for purposes like Firefox's > bookmarks and settings and such (which I think was what it was > originally developed for?). Not so fine over a network. The sheer number of Firefox bugs related to its use of SQLite says different. Once upon a time, Firefox's config, bookmarks, etc. were stored in plain text files. At worst they were HTML. You could trivially read them, copy them, restore them and even (if you were careful) edit them using the text editor of your choice. Many a time I was on one machine, wanted to know a bookmark from another machine, so I would ssh across to the other machine and run grep over the bookmark file. No more. Firefox still keeps a bookmark HTML file, but it never seems to be synced with the actual bookmarks. Settings are stored in an opaque blob, rather than human-readable text, limiting what you can do with it. It's very nice that Firefox offers about:config but not so nice that you can't do the same thing without the GUI running. If Firefox crashes, there are failure modes where it can no longer read your bookmarks, or keep history. I don't mean that history won't persist across restarts, I mean that *within a single session* it cannot remember what page you came from so you can hit the Back button and return to it. WTF? I swear, if not for the fact that every single other browser is worse, I would dump Firefox in a second. I don't believe for a second that moving to SQlite has anything to do with performance, because reading and writing preference settings should be rare and far from a bottleneck. SQlite is simply fragile and unreliable over a network, and people using their home directory on a network drive are not that rare. -- Steve From hayesstw at telkomsa.net Wed Feb 18 21:45:21 2015 From: hayesstw at telkomsa.net (Steve Hayes) Date: Thu, 19 Feb 2015 04:45:21 +0200 Subject: sqlite3 and dates References: Message-ID: On Wed, 18 Feb 2015 08:19:25 +0200, "Frank Millman" wrote: >Hi all > >sqlite3 does not have a DATE type, but the python module does a pretty good >job of providing one - The Rootsmagic genealogy program uses SQLite for its database, I don't know whether or to what extent it uses Python to interac t with the database, but it seems to do a pretty good job of handling dates, calculating ages etc. http://www.rootsmagic.com/ -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk From hayesstw at telkomsa.net Wed Feb 18 21:48:58 2015 From: hayesstw at telkomsa.net (Steve Hayes) Date: Thu, 19 Feb 2015 04:48:58 +0200 Subject: sqlite3 and dates References: Message-ID: <0ljaeal2vnr3gsuofo6gdetk2naf2oor0i@4ax.com> On Wed, 18 Feb 2015 22:21:35 +1100, Chris Angelico wrote: >On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer wrote: >> On 18.02.2015 08:05, Chris Angelico wrote: >> >>> But if you need more facilities than SQLite3 can offer, maybe it's >>> time to move up to a full database server, instead of local files. >>> Switching to PostgreSQL will give you all those kinds of features, >>> plus a lot of other things that I would have thought pretty basic - >>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >>> support that. >> >> I see you're running a lawnmower. Maybe you should switch to a combine >> harvester. That'll get you extra features like a reciprocating knife >> cutter bar. I was quite surprised that regular lawnmowers don't support >> those. > >SQLite3 is fine for something that's basically just a more structured >version of a flat file. You assume that nobody but you has the file >open, and you manipulate it just the same as if it were a big fat blob >of JSON, but thanks to SQLite, you don't have to rewrite the whole >file every time you make a small change. That's fine. But it's the >wrong tool for any job involving multiple users over a network, and >quite probably the wrong tool for a lot of other jobs too. It's the >smallest-end piece of software that can truly be called a database. I >would consider it to be the wrong database for serious accounting >work, and that's based on the ranting of a majorly-annoyed accountant >who had to deal with issues in professional systems that had made >similar choices in back-end selection. > >You're welcome to disagree, but since PostgreSQL doesn't cost any >money and (on Linux at least; can't speak for other platforms) doesn't >take significant effort to set up, I will continue to recommend it. All of which has nothing, absolutely nothing, to do with the OP's question, which said nothing about number of users, but how the software handles dates. -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk From hayesstw at telkomsa.net Wed Feb 18 21:53:23 2015 From: hayesstw at telkomsa.net (Steve Hayes) Date: Thu, 19 Feb 2015 04:53:23 +0200 Subject: Not sqlite3 and dates References: Message-ID: On Wed, 18 Feb 2015 23:14:32 +1100, Chris Angelico wrote: >On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer wrote: >> SQLite and Postgres are so vastly different in their setup, >> configuration, capabilities and requirements that the original developer >> has to have done a MAJOR error in judgement so that a change from one to >> the other would not be ill-advised. > >On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman wrote: >> My accounting software supports three databases - MS Sql Server, PostgreSQL, >> and sqlite3. > >Johannes, are you saying that Frank made three major errors of judgement? :) No, ChrisA did, in answering questions that no one was asking, and changing the subject of the thread without changing the subject line. -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk From hayesstw at telkomsa.net Wed Feb 18 21:54:11 2015 From: hayesstw at telkomsa.net (Steve Hayes) Date: Thu, 19 Feb 2015 04:54:11 +0200 Subject: Not sqlite3 and dates References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> Message-ID: On Thu, 19 Feb 2015 09:37:49 +1100, Chris Angelico wrote: >On Thu, Feb 19, 2015 at 9:17 AM, wrote: >>> SQLite3 is fine for something that's basically just a more structured >>> version of a flat file. You assume that nobody but you has the file >>> open, and you manipulate it just the same as if it were a big fat blob >>> of JSON, but thanks to SQLite, you don't have to rewrite the whole >>> file every time you make a small change. That's fine. >> >> That's bullshit. Sqlite offers a lot more than that including >> a SQL interface, transactions, referential integrity, constraints >> indexes, triggers and other general relational database features. >> >> That you would equate that to a JSON blob would indicate either >> a profound ignorance about Sqlite or (more likely) a need to >> defend your preference with complete disregard of fact. > >I didn't equate them. I said that SQLite3 is great if you look on it >as an upgrade over a JSON blob. Of course it offers more features than >that, and you don't need to swear at me to make your point. > >But SQLite3 is *not* great if you look on it as a database engine >comparable with DB2, PostgreSQL, and even MySQL. And how does that answer the OP's question? -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk From memilanuk at gmail.com Wed Feb 18 22:33:20 2015 From: memilanuk at gmail.com (memilanuk) Date: Wed, 18 Feb 2015 19:33:20 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <85lhjuahsm.fsf@benfinney.id.au> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: On 02/18/2015 04:03 PM, Ben Finney wrote: >> Is there anything *good* that sits in between the two extremes of >> SQLite and PostgreSQL? > > What do you need a RDBMS to do, and what do you not need? > > The answers to those questions vary hugely between different people (and > most people probably don't think too deeply about them). They will > determine what ?good? means for your case. > >> Is there nothing that amounts to a 'PostgreSQLite'? > > PostgreSQL itself fits that mould quite well; it is quite capable of > serving a small footprint while still offering full concurrency. > > I don't know of a free-software concurrent RDBMS which can be considered > lighter than that. (No, MySQL doesn't count; its concurrency is > *unreliable* and it commonly loses data silently. Don't use MySQL.) > > But perhaps you don't need concurrency? Only you can tell us. > At this point... I don't think concurrency is going to be a major requirement for what I have in mind. For one project, only a few people will be writing to the DB, and only by a stroke of luck would it be at the same time, and it would be very unlikely that they would be modifying the same record at the same time due to physical constraints. For the other... there may be anywhere from 1-10 (maybe more, but doubtful) entering data (creating new records for competitors, or entering existing competitors in a tournament). I have a hard time picturing that few people stressing a modern computer system enough to where SQLite couldn't keep up (thinking web-based interface using Flask or something similar). In the latter case, one of the over-arching priorities is that it be easily distributable, as in that people with relatively little knowledge of a database be able to set it up and run it. -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From breamoreboy at yahoo.co.uk Wed Feb 18 22:34:47 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 03:34:47 +0000 Subject: sqlite3 and dates In-Reply-To: <0ljaeal2vnr3gsuofo6gdetk2naf2oor0i@4ax.com> References: <0ljaeal2vnr3gsuofo6gdetk2naf2oor0i@4ax.com> Message-ID: On 19/02/2015 02:48, Steve Hayes wrote: > > All of which has nothing, absolutely nothing, to do with the OP's > question, which said nothing about number of users, but how the > software handles dates. > Very true, but charging off like this at massive tangents is one of the reasons I love being here. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Feb 18 22:43:36 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 03:43:36 +0000 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <54e546b5$0$11119$c3e8da3@news.astraweb.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> Message-ID: On 19/02/2015 02:13, Steven D'Aprano wrote: > Chris Angelico wrote: > >>> SQLite misses some important features that makes it better suited as a >>> simple datastore, not much unlike shelve. And network use is not one >>> of them, since you can actually implement concurrent sqlite access by >>> coding an intermediate layer. Assuming of course we are talking about >>> a small number of concurrent users. >> >> This is what I was saying: it's fine for purposes like Firefox's >> bookmarks and settings and such (which I think was what it was >> originally developed for?). Not so fine over a network. > > The sheer number of Firefox bugs related to its use of SQLite says > different. > > Once upon a time, Firefox's config, bookmarks, etc. were stored in plain > text files. At worst they were HTML. You could trivially read them, copy > them, restore them and even (if you were careful) edit them using the text > editor of your choice. Many a time I was on one machine, wanted to know a > bookmark from another machine, so I would ssh across to the other machine > and run grep over the bookmark file. > > No more. Firefox still keeps a bookmark HTML file, but it never seems to be > synced with the actual bookmarks. Settings are stored in an opaque blob, > rather than human-readable text, limiting what you can do with it. It's very > nice that Firefox offers about:config but not so nice that you can't do the > same thing without the GUI running. > > If Firefox crashes, there are failure modes where it can no longer read your > bookmarks, or keep history. I don't mean that history won't persist across > restarts, I mean that *within a single session* it cannot remember what page > you came from so you can hit the Back button and return to it. WTF? > > I swear, if not for the fact that every single other browser is worse, I > would dump Firefox in a second. > After a wonderful relationship lasting many happy years I dumped Firefox a few weeks ago for Chrome. A few anxious moments gave me pause for thought, but overall I'm happy to have changed. However is anybody aware of a "new kid on the block" that could take over as I'd happily switch again? Nothing has sprung out at me, hence the choice I made. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From shalini.ravishankar at gmail.com Wed Feb 18 22:55:58 2015 From: shalini.ravishankar at gmail.com (Shalini Ravishankar) Date: Wed, 18 Feb 2015 19:55:58 -0800 (PST) Subject: VTK Quadratic - vtkContourFilter - vtkSliderRepresentation2D in Python not working properly Message-ID: <5be50237-c500-42f9-a469-a7afab4fb50a@googlegroups.com> Hello Everyone, I am new to VTK. I am trying to extract isosurfaces(Contour) from a quadratic function with a Slider to change the iso surfaces. #!/usr/bin/env python # First, we need to import vtk package in order to access VTK classes/functions. import vtk # create a data source...an implicit function. quadric = vtk.vtkQuadric() quadric.SetCoefficients(.5, 1, .2, 0, .1, 0, 0, .2, 0, 0) def vtkSliderCallback2(obj, event): sliderRepres = obj.GetRepresentation() pos = sliderRepres.GetValue() print "Position ",pos isosurface.SetValue(0, pos) # create a filter...a sampling function, which samples an implicit function over the x-y-z range # although this object is not called "filter" it takes an input and do something to/with it # and produce an output. sample = vtk.vtkSampleFunction() sample.SetSampleDimensions(100, 100, 100) sample.SetImplicitFunction(quadric) outline = vtk.vtkOutlineFilter() outline.SetInput( sample.GetOutput() ) outlineMapper = vtk.vtkPolyDataMapper() outlineMapper.SetInput( outline.GetOutput() ) outlineActor = vtk.vtkActor() outlineActor.SetMapper( outlineMapper ) outlineActor.GetProperty().SetColor(0.0,0.0,1.0) # create another filter...computing a contour of an input data. isosurface = vtk.vtkContourFilter() isosurface.SetInputConnection(sample.GetOutputPort()) isosurface.GenerateValues(15, 0.0, 4.2) # create a mapper, which mapps data to visualizable data structure. contMapper = vtk.vtkPolyDataMapper() contMapper.SetInputConnection(isosurface.GetOutputPort()) contMapper.SetScalarRange(0.0, 1.2) # create an actor, which can be displayed. contActor = vtk.vtkActor() contActor.SetMapper(contMapper) # create a window with a renderer. ren = vtk.vtkRenderer() renWin = vtk.vtkRenderWindow() renWin.AddRenderer(ren) iren = vtk.vtkRenderWindowInteractor() iren.SetRenderWindow(renWin) ren.SetBackground(0.95, 0.95, 1.0) ren.AddActor(contActor) renWin.SetSize( 500, 500 ) SliderRepres = vtk.vtkSliderRepresentation2D() min = 0 #ImageViewer.GetSliceMin() max = 256 #ImageViewer.GetSliceMax() SliderRepres.SetMinimumValue(min) SliderRepres.SetMaximumValue(max) SliderRepres.SetValue((min + max) / 2) SliderRepres.SetTitleText("Slider") SliderRepres.GetPoint1Coordinate().SetCoordinateSystemToNormalizedDisplay() SliderRepres.GetPoint1Coordinate().SetValue(0.2, 0.9) SliderRepres.GetPoint2Coordinate().SetCoordinateSystemToNormalizedDisplay() SliderRepres.GetPoint2Coordinate().SetValue(0.8, 0.9) SliderRepres.SetSliderLength(0.02) SliderRepres.SetSliderWidth(0.03) SliderRepres.SetEndCapLength(0.01) SliderRepres.SetEndCapWidth(0.03) SliderRepres.SetTubeWidth(0.005) SliderRepres.SetLabelFormat("%3.0lf") SliderRepres.SetTitleHeight(0.02) SliderRepres.SetLabelHeight(0.02) SliderWidget = vtk.vtkSliderWidget() SliderWidget.SetInteractor(iren) SliderWidget.SetRepresentation(SliderRepres) SliderWidget.KeyPressActivationOff() SliderWidget.SetAnimationModeToAnimate() SliderWidget.SetEnabled(True) SliderWidget.AddObserver("EndInteractionEvent", vtkSliderCallback2) # this causes the pipeline to "execute" renWin.Render() # initialize and start the interactor iren.Initialize() iren.Start() This is my code. It gives me output for the quadratic fucntion but when I change the contour values using slider I couldn't see the changes. Can Someone tell me What I am doing wrong here?? From rosuav at gmail.com Wed Feb 18 23:01:23 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 15:01:23 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: On Thu, Feb 19, 2015 at 2:33 PM, memilanuk wrote: > At this point... I don't think concurrency is going to be a major > requirement for what I have in mind. For one project, only a few people > will be writing to the DB, and only by a stroke of luck would it be at the > same time, and it would be very unlikely that they would be modifying the > same record at the same time due to physical constraints. > > For the other... there may be anywhere from 1-10 (maybe more, but doubtful) > entering data (creating new records for competitors, or entering existing > competitors in a tournament). I have a hard time picturing that few people > stressing a modern computer system enough to where SQLite couldn't keep up > (thinking web-based interface using Flask or something similar). In the > latter case, one of the over-arching priorities is that it be easily > distributable, as in that people with relatively little knowledge of a > database be able to set it up and run it. Both of these need concurrency. You may not need _heavy_ concurrency, but you certainly do need to cope adequately with multiple simultaneous users. Your first case is a perfect example of why you need a database rather than flat files; in fact, you want the granularity of record-level locking rather than table-level. Alas, SQLite3 does not actually offer this (in fact, I'm not sure it even offers table-level locking); once any process begins writing to the database, all others are locked out (even for reading) until it finishes. That's fine if you (a) don't write very often, and (b) don't write very much, but the fact that you're trying to modify different records doesn't help you here. It does with full-scale database systems, where you actually do have record-level locking, but not with SQLite3. Your second case definitely demands concurrency. I've seen tournaments for various games where database-level write locking would be a critical problem, and that with only a couple hundred players and a handful of people keying in data. Of course, it depends how much effort it takes to key that in. If the humans have to enter extensive reports on the tournament results, they'll spend most of their time doing that; but if their job is to quickly say "X beat Y 2-1" and then get back results saying "X plays Z next, Y gets a bye", then you need your database to react quickly, even if three other people are entering results. So it's a huge question of human versus computer workload... but once again, chances are you need record-level locking. It may very well turn out that SQLite3 is entirely capable of the job. But it's certainly not proven by your above statements, and I would start by assuming PostgreSQL by default. ChrisA From rurpy at yahoo.com Wed Feb 18 23:05:50 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 20:05:50 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: <42b9ed87-a40a-4e70-afd3-304dc55fa0f0@googlegroups.com> On 02/18/2015 05:08 PM, Mario Figueiredo wrote: >[...] > SQLite misses some important features that makes it better suited as a > simple datastore, not much unlike shelve. And network use is not one > of them, since you can actually implement concurrent sqlite access by > coding an intermediate layer. Assuming of course we are talking about > a small number of concurrent users. I think there are some persistent misunderstandings about Sqlite in this thread, Sqlite offers concurrent access already. What Sqlite doesn't offer is high performance concurrent write access. That is, it locks the entire database for the duration of a write operation. Given that most such operations are pretty short, for a small number of concurrent writers this is not a big problem. From rurpy at yahoo.com Wed Feb 18 23:08:03 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 20:08:03 -0800 (PST) Subject: sqlite3 and dates In-Reply-To: <54e51b29$0$12994$c3e8da3$5496439d@news.astraweb.com> References: <9799edb6-d01d-49c2-82a4-3d70d9ca2427@googlegroups.com> <54e51b29$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5fff723b-c980-4746-bb24-f54a15392747@googlegroups.com> On 02/18/2015 04:07 PM, Steven D'Aprano wrote: > rurpy at yahoo.com wrote: >> On 02/18/2015 01:14 PM, Ben Finney wrote: >>> Johannes Bauer writes: >>>> On 18.02.2015 08:05, Chris Angelico wrote: >>>> >>>>> But if you need more facilities than SQLite3 can offer, maybe it's >>>>> time to move up to a full database server, instead of local files. >>>>> Switching to PostgreSQL will give you all those kinds of features, >>>>> plus a lot of other things that I would have thought pretty basic - >>>>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't >>>>> support that. >>>> >>>> I see you're running a lawnmower. Maybe you should switch to a combine >>>> harvester. That'll get you extra features like a reciprocating knife >>>> cutter bar. I was quite surprised that regular lawnmowers don't support >>>> those. >>> >>> Chris has pointed out one flaw in this analogy; I'll address another. >>> >>> A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife >>> cutter bar". I'm in agreement that it is a pretty basic SQL feature, and >>> it doesn't appear to conflict with the narrow focus that we all agree is >>> appropriate for SQLite. >> >> No, you and Chris are way off base and Johannes is correct. >> He was pointing out that there are many applications that can >> benefit from a database and a full-blown, bells and whistles >> solution like Postgresql is often overkill in that (very common) >> case. His analogy is quite apt and I wish I'd thought of it. > > > I'm not seeing that at all. Chris explicitly proceeded his comments with the > condition "if you need more facilities than SQLite3 can offer". Right. And did so in a context where there the facility presumed not to be offered was getting a date back from Sqlite. Common sense should tell anyone that it is very improbable that there is no way to get a date out of a sqlite database. Chris' "solution" to that problem? Switch to Postgresql. That was the context for Johannes' analogy. > Johannes' > analogy ignores that and consequently mocks the very idea that anyone might > need more than a regular lawmower -- even a farmer with a thousand acres of > wheat to be harvested. The analogy works precisely because farmers with a thousand acres of wheat to be harvested need reciprocating knife cutter bars. In the same way people dealing with terabytes of data, concurrent updates, enterprise scale data and applications need a Postregsql. People mowing their lawns do not. People without needs for the things that client server database do well, whose problem is not immediately being able to figure out how to get a date, do not. (Did you really need that explained to you?) If you didn't interpret it the way I did (and the way I presume Johannes meant it) then, well, you didn't interpret it the same. Your interpretation (especially given your penchant for sophistry) are not my problem. I am quite happy to let anyone reading make their own evaluation. > Johannes' subsequent posts are more nuanced about Sqlite filling a niche and > not being suitable for everything, but the analogy you're supporting > doesn't. It's an amusing quip, quite funny, but like most quips, lacks > nuance and misrepresents Chris' original position. All analogies are imperfect and thus make easy targets. Shrug. >[...] From ben+python at benfinney.id.au Wed Feb 18 23:09:56 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 15:09:56 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: <858ufua6ej.fsf@benfinney.id.au> memilanuk writes: > At this point... I don't think concurrency is going to be a major > requirement for what I have in mind. What's the difference betwen a ?requirement? and a ?major requirement?? If you want networked access, you need concurrent access and access permissions, etc. SQLite does not have concurrent access. Once you require concurrent access, you need something more complex, like PostgreSQL. > I have a hard time picturing that few people stressing a modern > computer system enough to where SQLite couldn't keep up (thinking > web-based interface using Flask or something similar). In the latter > case, one of the over-arching priorities is that it be easily > distributable, as in that people with relatively little knowledge of a > database be able to set it up and run it. Set it up where? Are you hoping that a network-accessible service can be set up without knowledge of the specific concurrent authenticated networked access is needed in each installation? -- \ ?Broken promises don't upset me. I just think, why did they | `\ believe me?? ?Jack Handey | _o__) | Ben Finney From rurpy at yahoo.com Wed Feb 18 23:10:14 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 20:10:14 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <54e546b5$0$11119$c3e8da3@news.astraweb.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> Message-ID: On 02/18/2015 07:13 PM, Steven D'Aprano wrote:> Chris Angelico wrote: >>> SQLite misses some important features that makes it better suited as a >>> simple datastore, not much unlike shelve. And network use is not one >>> of them, since you can actually implement concurrent sqlite access by >>> coding an intermediate layer. Assuming of course we are talking about >>> a small number of concurrent users. >> >> This is what I was saying: it's fine for purposes like Firefox's >> bookmarks and settings and such (which I think was what it was >> originally developed for?). Not so fine over a network. > > The sheer number of Firefox bugs related to its use of SQLite says > different. > > Once upon a time, Firefox's config, bookmarks, etc. were stored in plain > text files. At worst they were HTML. You could trivially read them, copy > them, restore them and even (if you were careful) edit them using the text > editor of your choice. Many a time I was on one machine, wanted to know a > bookmark from another machine, so I would ssh across to the other machine > and run grep over the bookmark file. I agree, I prefer plain text files whenever practical. But since the original discussion was about Sqlite vs Postgresql, not Sqlite vs text files, shouldn't the question be: would Firefox be better if it required you to install and configure Postgreql instead of using Sqlite? > No more. Firefox still keeps a bookmark HTML file, but it never seems to be > synced with the actual bookmarks. Settings are stored in an opaque blob, > rather than human-readable text, limiting what you can do with it. It's very > nice that Firefox offers about:config but not so nice that you can't do the > same thing without the GUI running. > > If Firefox crashes, there are failure modes where it can no longer read your > bookmarks, or keep history. I don't mean that history won't persist across > restarts, I mean that *within a single session* it cannot remember what page > you came from so you can hit the Back button and return to it. WTF? > > I swear, if not for the fact that every single other browser is worse, I > would dump Firefox in a second. > > I don't believe for a second that moving to SQlite has anything to do with > performance, because reading and writing preference settings should be rare > and far from a bottleneck. SQlite is simply fragile and unreliable over a > network, and people using their home directory on a network drive are not > that rare. I don't see any evidence that it is Sqlite that is the problem as opposed to FF's use (or misuse) of it, or other problems that are in FF and have nothing to do with Sqlite. If Sqlite reliably implements ACID semantics as they claim, is certainly should be possible to make use of it without the problems you (and I too) see. And there is no reason to believe the situation would be any better with Postgresql. From ethan at stoneleaf.us Wed Feb 18 23:15:30 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 20:15:30 -0800 Subject: When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)] In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: <54E56362.3000208@stoneleaf.us> At the risk of using actual data, I looked this up at http://www.sqlite.org/whentouse.html: Checklist For Choosing The Right Database Engine * Is the data separated from the application by a network? ? choose client/server Relational database engines act as a bandwidth-reducing data filter. So it is best to keep the database engine and the data on the same physical device so that the high-bandwidth engine-to-disk link does not have to traverse the network, only the lower-bandwidth application-to-engine link. But SQLite is built into the application. So if the data is on a separate device from the application, it is required that the higher bandwidth engine-to-disk link be across the network. This works, but it is suboptimal. Hence, it is usually better to select a client/server database engine when the data is on a separate device from the application. * Many concurrent writers? ? choose client/server If many threads and/or processes need to write the database at the same instant (and they cannot queue up and take turns) then it is best to select a database engine that supports that capability, which always means a client/server database engine. SQLite only supports one writer at a time per database file. But in most cases, a write transaction only takes milliseconds and so multiple writers can simply take turns. SQLite will handle more write concurrency that many people suspect. Nevertheless, client/server database systems, because they have a long-running server process at hand to coordinate access, can usually handle far more write concurrency than SQLite ever will. * Big data? ? choose client/server If your data will grow to a size that you are uncomfortable or unable to fit into a single disk file, then you should select a solution other than SQLite. SQLite supports databases up to 140 terabytes in size, assuming you can find a disk drive and filesystem that will support 140-terabyte files. Even so, when the size of the content looks like it might creep into the terabyte range, it would be good to consider a centralized client/server database. * Otherwise ? choose SQLite! For device-local storage with low writer concurrency and less than a terabyte of content, SQLite is almost always a better solution. SQLite is fast and reliable and it requires no configuration or maintenance. It keeps thing simple. SQLite "just works". -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rurpy at yahoo.com Wed Feb 18 23:26:23 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Wed, 18 Feb 2015 20:26:23 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: On 02/18/2015 09:09 PM, Ben Finney wrote > memilanuk writes: >[...] > If you want networked access, you need concurrent access and access > permissions, etc. Sqlite has concurrent access. It doesn't have concurrent access that will support a large number of writers or high volume of writes. As for access permissions, it is common, even with Postgresql to do all database access through a single Postgresql user and to implement authorization and access permission in the application. > SQLite does not have concurrent access. Once you require concurrent > access, you need something more complex, like PostgreSQL. Please read https://www.sqlite.org/faq.html#q5 >[...] From memilanuk at gmail.com Wed Feb 18 23:29:38 2015 From: memilanuk at gmail.com (memilanuk) Date: Wed, 18 Feb 2015 20:29:38 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <858ufua6ej.fsf@benfinney.id.au> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> Message-ID: On 02/18/2015 08:09 PM, Ben Finney wrote: >> I have a hard time picturing that few people stressing a modern >> computer system enough to where SQLite couldn't keep up (thinking >> web-based interface using Flask or something similar). In the latter >> case, one of the over-arching priorities is that it be easily >> distributable, as in that people with relatively little knowledge of a >> database be able to set it up and run it. > > Set it up where? Are you hoping that a network-accessible service can be > set up without knowledge of the specific concurrent authenticated > networked access is needed in each installation? > They would need to be able to set up the application (and whatever database) on their laptop or PC, wherever that may be, and spend their time administering the event, not the database engine. Once its set, it shouldn't need any tending, or they are going to be SOL as I wouldn't be able to help them. It may be that Flask + SQLite will be enough; otherwise I foresee a disproportional amount of *my* time will be spent documenting and explaining how to set up and maintain a RDBMS on Windows, on a Mac, etc. Starting to wonder if a pre-configured VM appliance running in Virtualbox might be simpler for the end user to set up and run. -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From ben+python at benfinney.id.au Wed Feb 18 23:36:17 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 15:36:17 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> Message-ID: <854mqia56m.fsf@benfinney.id.au> memilanuk writes: > They would need to be able to set up the application (and whatever > database) on their laptop or PC, wherever that may be, and spend their > time administering the event, not the database engine. So, the database will only be accessed by exactly one application, on exactly the same machine and storage as the application? If so, you don't need concurrency. Otherwise, your database needs concurrency; and the person installing the database will need to make a lot of decisions about the specific network environment and devices to be allowed to access the database. But is this what you mean by your requirements not being met by SQLite? -- \ ?Natural catastrophes are rare, but they come often enough. We | `\ need not force the hand of nature.? ?Carl Sagan, _Cosmos_, 1980 | _o__) | Ben Finney From jsf80238 at gmail.com Wed Feb 18 23:44:12 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Wed, 18 Feb 2015 21:44:12 -0700 Subject: What behavior would you expect? Message-ID: I have need to search a directory and return the name of the most recent file matching a given pattern. Given a directory with these files and timestamps, q.pattern1.abc Feb 13 r.pattern1.cdf Feb 12 s.pattern1.efg Feb 10 t.pattern2.abc Feb 13 u.pattern2.xyz Feb 14 v.pattern2.efg Feb 10 calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc and calling my_function("/path/to/dir", "pattern2") will return u.pattern2.xyz. My question is, what would be a reasonable behavior/result/return value if: 1. "/path/to/dir" does not exist or is not readable 2. no files match the given pattern Also, what would be a reasonable name for such a function? -------------- next part -------------- An HTML attachment was scrubbed... URL: From memilanuk at gmail.com Wed Feb 18 23:57:04 2015 From: memilanuk at gmail.com (memilanuk) Date: Wed, 18 Feb 2015 20:57:04 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <854mqia56m.fsf@benfinney.id.au> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> <854mqia56m.fsf@benfinney.id.au> Message-ID: On 02/18/2015 08:36 PM, Ben Finney wrote: > memilanuk writes: > >> They would need to be able to set up the application (and whatever >> database) on their laptop or PC, wherever that may be, and spend their >> time administering the event, not the database engine. > > So, the database will only be accessed by exactly one application, on > exactly the same machine and storage as the application? If so, you > don't need concurrency. > > Otherwise, your database needs concurrency; and the person installing > the database will need to make a lot of decisions about the specific > network environment and devices to be allowed to access the database. > > But is this what you mean by your requirements not being met by SQLite? > In the past I've been waffling back and forth between a desktop client/server setup, or a web-based interface with everything on one computer. At this point I'm leaning toward the latter. -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From lac at openend.se Wed Feb 18 23:57:31 2015 From: lac at openend.se (Laura Creighton) Date: Thu, 19 Feb 2015 05:57:31 +0100 Subject: can python handle CHIME .spt files? In-Reply-To: Message from Davin Potts of "Wed, 18 Feb 2015 15:55:03 -0600." <386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU><386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> Message-ID: <201502190457.t1J4vVSh030332@fido.openend.se> I went and asked your question to Andrew Dalke, who is an expert in such things. Writing programs for visualising molecules is what he does for a living. The news is not good. First of all, he says the "spt" format is a command format. See http://www.callutheran.edu/BioDev/omm/scripting_ch/molmast.htm for examples. select all; wireframe; dots off; wireframe off; backbone; wireframe 100; spacefill off; list molsurface transparent; select all; spacefill; set specular on; set specpower 20; define piece :A and 10, :A and 11, :A and 12; select all; spacefill off; wireframe; select atomno=4174; label protein; It's like Tcl, with lots of internal syntax handled by the individual commands. For example, "select atomno=4174" takes a full expression, like select atomno >= 195 and atomno <= 277 More details at http://jmol.sourceforge.net/docs/JmolUserGuide/ch04.html . The only way to tell that it works, in Python and without using Chime, is to re-write command interpreter that understands the script syntax. Andrew Dalke says that he tried something like that back in the 1990s, and while it's possible, it is not easy. When Andrew says things like that, you can bet money that it is very, very hard, because he does seemingly impossible things for fun over breakfast. Indeed, I wrote him hoping he would have scripts lying around that already do this ... He says that, these days, the best solution would probably be some in-browser Javascript code that drives Chime and checks if there's an error, but this is also not a job for somebody who is learning how to program, and iff it's less than several dozen/100 scripts, then really this is best done manually, unless the person enjoys programming. If you want to write him and ask him more questions, his contact page is at http://www.dalkescientific.com/contact.html He's very friendly, as well as being a very good friend of mine. Laura Creighton From hayesstw at telkomsa.net Wed Feb 18 23:59:53 2015 From: hayesstw at telkomsa.net (Steve Hayes) Date: Thu, 19 Feb 2015 06:59:53 +0200 Subject: When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)] References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: On Wed, 18 Feb 2015 20:15:30 -0800, Ethan Furman wrote: >At the risk of using actual data, I looked this up at http://www.sqlite.org/whentouse.html: > > >Checklist For Choosing The Right Database Engine Interesting. A couple of months ago I asked in comp.databases what the differences were between SQLite and MySQL, and I got a lot of uninformative gobbledegook. This was more informative. I would summarise it by saying if you want a multiuser database running on a network, use MySQL. If you want a standalone database on a single machine, use SQLite. -- Steve Hayes from Tshwane, South Africa Web: http://www.khanya.org.za/stevesig.htm Blog: http://khanya.wordpress.com E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk From ethan at stoneleaf.us Thu Feb 19 00:07:47 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 21:07:47 -0800 Subject: When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)] In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: <54E56FA3.7080504@stoneleaf.us> On 02/18/2015 08:59 PM, Steve Hayes wrote: > I would summarise it by saying [...] if you want a standalone database on > a single machine, use SQLite. It sounds like SQLite would also work fine if that single-machine scenario was a web-app with not-too-many users trying to write at once. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rosuav at gmail.com Thu Feb 19 00:10:02 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 16:10:02 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 3:44 PM, Jason Friedman wrote: > I have need to search a directory and return the name of the most recent > file matching a given pattern. Given a directory with these files and > timestamps, > > q.pattern1.abc Feb 13 > r.pattern1.cdf Feb 12 > s.pattern1.efg Feb 10 > t.pattern2.abc Feb 13 > u.pattern2.xyz Feb 14 > v.pattern2.efg Feb 10 > > calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc > and calling my_function("/path/to/dir", "pattern2") will return > u.pattern2.xyz. That seems reasonable, and well-defined. > My question is, what would be a reasonable behavior/result/return value if: > 1. "/path/to/dir" does not exist or is not readable Raise an exception. Or, better still, just allow the exception to bubble. > 2. no files match the given pattern Either return None, or raise an exception, depending on how "normal" this state is. Is it simply a matter of "you asked for something, but you got nothing"? Then return None. Is it a really unusual situation that should basically never happen? Then raise an exception, so you get an instant report with no wondering "why am I getting these strange TypeErrors". > Also, what would be a reasonable name for such a function? newest_file_matching() seems decent. Remember, it doesn't have to be especially short. ChrisA From ben+python at benfinney.id.au Thu Feb 19 00:16:29 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 19 Feb 2015 16:16:29 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> <854mqia56m.fsf@benfinney.id.au> Message-ID: <85wq3e8or6.fsf@benfinney.id.au> memilanuk writes: > In the past I've been waffling back and forth between a desktop > client/server setup, or a web-based interface with everything on one > computer. At this point I'm leaning toward the latter. So, it's been many exchanges back and forth, and you still aren't telling us what specific needs you have that SQLite can't provide. At this point I'm just going to have to wait until you can lay out the specifics. -- \ ?In economics, hope and faith coexist with great scientific | `\ pretension and also a deep desire for respectability.? ?John | _o__) Kenneth Galbraith, 1970-06-07 | Ben Finney From greg.ewing at canterbury.ac.nz Thu Feb 19 00:22:57 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 19 Feb 2015 18:22:57 +1300 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: Mario Figueiredo wrote: > Parameterized queries is just a pet peeve of mine that I wish to > include here. SQLite misses it How does sqlite3 miss parameterized queries? It supports DB-API parameter subsitution with '?' according to the docs. -- Greg From ethan at stoneleaf.us Thu Feb 19 00:23:50 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 21:23:50 -0800 Subject: can python handle CHIME .spt files? In-Reply-To: <201502190457.t1J4vVSh030332@fido.openend.se> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU><386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> <201502190457.t1J4vVSh030332@fido.openend.se> Message-ID: <54E57366.200@stoneleaf.us> On 02/18/2015 08:57 PM, Laura Creighton wrote: > I went and asked your question to Andrew Dalke, who is an expert > in such things. Did you happen to ask him about PyMol? Just curious. ;) -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From memilanuk at gmail.com Thu Feb 19 00:26:37 2015 From: memilanuk at gmail.com (memilanuk) Date: Wed, 18 Feb 2015 21:26:37 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <85wq3e8or6.fsf@benfinney.id.au> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> <854mqia56m.fsf@benfinney.id.au> <85wq3e8or6.fsf@benfinney.id.au> Message-ID: On 02/18/2015 09:16 PM, Ben Finney wrote: > memilanuk writes: > >> In the past I've been waffling back and forth between a desktop >> client/server setup, or a web-based interface with everything on one >> computer. At this point I'm leaning toward the latter. > > So, it's been many exchanges back and forth, and you still aren't > telling us what specific needs you have that SQLite can't provide. At > this point I'm just going to have to wait until you can lay out the > specifics. > Okay, let me put it like this: if I set up a web interface using Flask for the front-end, and SQLite as the backend DB, running from a PC/laptop, with anywhere from 1 to 10 people doing data entry from other devices (laptops, desktops, tablets, etc.) at roughly the same time, is SQLite going to be 'concurrent' enough? -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From breamoreboy at yahoo.co.uk Thu Feb 19 00:32:31 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 05:32:31 +0000 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: On 19/02/2015 00:08, Mario Figueiredo wrote: > > Parameterized queries is just a pet peeve of mine that I wish to > include here. SQLite misses it and I miss the fact SQLite misses it. > The less SQL one needs to write in their code, the happier one should > be. > https://docs.python.org/3/library/sqlite3.html#module-sqlite3 paragraphs seven and eight. "Usually your SQL operations will need to use values from Python variables. You shouldn?t assemble your query using Python?s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong). Instead, use the DB-API?s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor?s execute() method. (Other database modules may use a different placeholder, such as %s or :1.) For example:..." -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ethan at stoneleaf.us Thu Feb 19 00:37:43 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 18 Feb 2015 21:37:43 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> <854mqia56m.fsf@benfinney.id.au> <85wq3e8or6.fsf@benfinney.id.au> Message-ID: <54E576A7.3010804@stoneleaf.us> On 02/18/2015 09:26 PM, memilanuk wrote: > On 02/18/2015 09:16 PM, Ben Finney wrote: >> memilanuk writes: >> >>> In the past I've been waffling back and forth between a desktop >>> client/server setup, or a web-based interface with everything on one >>> computer. At this point I'm leaning toward the latter. >> >> So, it's been many exchanges back and forth, and you still aren't >> telling us what specific needs you have that SQLite can't provide. At >> this point I'm just going to have to wait until you can lay out the >> specifics. >> > > Okay, let me put it like this: if I set up a web interface using Flask for the front-end, and SQLite as the backend DB, > running from a PC/laptop, with anywhere from 1 to 10 people doing data entry from other devices (laptops, desktops, > tablets, etc.) at roughly the same time, is SQLite going to be 'concurrent' enough? Well, having zero experience with SQLite, but having read the docs just today [snip snide remark] -- I think you'll be fine with SQLite under those conditions. :) -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From davea at davea.name Thu Feb 19 01:29:06 2015 From: davea at davea.name (Dave Angel) Date: Thu, 19 Feb 2015 01:29:06 -0500 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: <54E582B2.6080203@davea.name> On 02/19/2015 12:10 AM, Chris Angelico wrote: > On Thu, Feb 19, 2015 at 3:44 PM, Jason Friedman wrote: >> I have need to search a directory and return the name of the most recent >> file matching a given pattern. Given a directory with these files and >> timestamps, >> >> q.pattern1.abc Feb 13 >> r.pattern1.cdf Feb 12 >> s.pattern1.efg Feb 10 >> t.pattern2.abc Feb 13 >> u.pattern2.xyz Feb 14 >> v.pattern2.efg Feb 10 >> >> calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc >> and calling my_function("/path/to/dir", "pattern2") will return >> u.pattern2.xyz. > > That seems reasonable, and well-defined. > >> My question is, what would be a reasonable behavior/result/return value if: >> 1. "/path/to/dir" does not exist or is not readable > > Raise an exception. Or, better still, just allow the exception to bubble. > >> 2. no files match the given pattern > > Either return None, or raise an exception, depending on how "normal" > this state is. Is it simply a matter of "you asked for something, but > you got nothing"? Then return None. Is it a really unusual situation > that should basically never happen? Then raise an exception, so you > get an instant report with no wondering "why am I getting these > strange TypeErrors". > >> Also, what would be a reasonable name for such a function? > > newest_file_matching() seems decent. Remember, it doesn't have to be > especially short. > Consider returning a list of files, sorted by datestamp. Return an empty list if there are no matches. That way, the user can find the newest, the oldest, can check if any match, and so on, all by how it manipulates the list. if result: check whether there are any matches newest = result[0] oldest = result[-1] Your code probably has to build the list anyway, to find the newest. So why not just return it, and let the caller decide which part(s) to keep. -- DaveA From lac at openend.se Thu Feb 19 02:04:26 2015 From: lac at openend.se (Laura Creighton) Date: Thu, 19 Feb 2015 08:04:26 +0100 Subject: can python handle CHIME .spt files? In-Reply-To: Message from Ethan Furman of "Wed, 18 Feb 2015 21:23:50 -0800." <54E57366.200@stoneleaf.us> References: <26AE89FA3A4C214D9E3F06D0AEA41A4083009639@EXCH10MBX01.CAMPUS.CU.CLEMSON.EDU><386723F8-673F-40C8-93BB-24ABE8EEA6EB@discontinuity.net> <201502190457.t1J4vVSh030332@fido.openend.se><54E57366.200@stoneleaf.us> Message-ID: <201502190704.t1J74QZP024060@fido.openend.se> In a message of Wed, 18 Feb 2015 21:23:50 -0800, Ethan Furman writes: >On 02/18/2015 08:57 PM, Laura Creighton wrote: > >> I went and asked your question to Andrew Dalke, who is an expert >> in such things. > >Did you happen to ask him about PyMol? Just curious. ;) > >-- >~Ethan~ I hadn't then, but have since. I will report what he says. Laura From steve+comp.lang.python at pearwood.info Thu Feb 19 02:07:17 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 19 Feb 2015 18:07:17 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> Message-ID: <54e58ba8$0$11121$c3e8da3@news.astraweb.com> rurpy at yahoo.com wrote: > On 02/18/2015 07:13 PM, Steven D'Aprano wrote:> Chris Angelico wrote: >>>> SQLite misses some important features that makes it better suited as a >>>> simple datastore, not much unlike shelve. And network use is not one >>>> of them, since you can actually implement concurrent sqlite access by >>>> coding an intermediate layer. Assuming of course we are talking about >>>> a small number of concurrent users. >>> >>> This is what I was saying: it's fine for purposes like Firefox's >>> bookmarks and settings and such (which I think was what it was >>> originally developed for?). Not so fine over a network. >> >> The sheer number of Firefox bugs related to its use of SQLite says >> different. >> >> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain >> text files. At worst they were HTML. You could trivially read them, copy >> them, restore them and even (if you were careful) edit them using the >> text editor of your choice. Many a time I was on one machine, wanted to >> know a bookmark from another machine, so I would ssh across to the other >> machine and run grep over the bookmark file. > > I agree, I prefer plain text files whenever practical. But since > the original discussion was about Sqlite vs Postgresql, not Sqlite > vs text files, shouldn't the question be: would Firefox be better > if it required you to install and configure Postgreql instead of > using Sqlite? Very possibly. With modern dependency management, it isn't hard to install Postgresql: sudo aptitude postgresql or equivalent should work. For primitive operating systems with no dependency management available, Firefox could come with a simple script which downloads, installs, configures and runs Postgresql. (Everything is simple for the guy who doesn't have to do it.) Possible snags: - Possibly postgresql is simply *too big*. Your 50MB(?) Firefox turns into a 2GB install. I doubt it -- on Debian, postgresql is 17MB installed. But I don't know what dependencies I'm not counting. - Or it is impossible to configure without excessive amounts of tech-savvy human intervention. Again, I doubt it. I seem to recall needing to create a Postgresql user and password. But maybe even that is too technical for the average Firefox user. - Maybe there are nasty interactions between Postgresql listening on some port and Windows firewall wanting to block that same port. Or... and here is a radical thought... maybe Firefox could give you the choice of which database? By default, it might use Sqlite, to satisfy the desktop users who don't want to think about it. And for those who are disturbed by the fragility of Sqlite on a network home directory, you just set a config setting in about:config to point at your existing Postgresql instance, and never need worry about it again. The Firefox devs surprise and confuse me. On the one hand, they have designed a powerful plug-in architecture, and encourage their user-base to use it for all sorts of amazing functionality that they don't want to build into the core browser. Yay for this. And on the other hand, they are *actively hostile* to any suggestion that using SQlite is not the best and *only* appropriate solution to the problem of storing config, bookmarks and history. A plug-in database architecture would probably work really well. > I don't see any evidence that it is Sqlite that is the problem > as opposed to FF's use (or misuse) of it, or other problems that > are in FF and have nothing to do with Sqlite. No no, even Sqlite devs recommend against using it on network drives. The Firefox problem is that when FF crashes, as it can do, or if you yank the power to the computer and everything dies, if your home directory is on a network drive, the database may be left in a locked state, or even corrupted. Nothing that the FF developers can do, given the choice of Sqlite. > If Sqlite reliably > implements ACID semantics as they claim, Ah, well "reliably" is a tricky word... http://stackoverflow.com/questions/788517/sqlite-over-a-network-share -- Steve From steve+comp.lang.python at pearwood.info Thu Feb 19 02:23:14 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 19 Feb 2015 18:23:14 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> Message-ID: <54e58f63$0$12975$c3e8da3$5496439d@news.astraweb.com> Ben Finney wrote: > What's the difference betwen a ?requirement? and a ?major requirement?? "I require a gold-plated Mercedes, if I can have it for less than $30,000. My major requirement is for some sort of personal transport." *wink* -- Steve From rosuav at gmail.com Thu Feb 19 02:23:26 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 18:23:26 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <54e58ba8$0$11121$c3e8da3@news.astraweb.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On Thu, Feb 19, 2015 at 6:07 PM, Steven D'Aprano wrote: > Very possibly. With modern dependency management, it isn't hard to install > Postgresql: > > sudo aptitude postgresql > > or equivalent should work. For primitive operating systems with no > dependency management available, Firefox could come with a simple script > which downloads, installs, configures and runs Postgresql. (Everything is > simple for the guy who doesn't have to do it.) Definitely a possibility. I'm pretty sure I've seen that exact thing done by one application (on Windows; on a typical Linux system, that'll be done by simple dependency management - your package metadata says "depends on postgresql" and apt-get or yum or whatever will do the work), and it's quite common for a Windows installer wizard to go through a series of subcomponents (grab the .NET framework, grab these hotfixes that the program depends on, grab some adware toolbar that you forgot to untick, and *then* install the program you asked for). > Possible snags: > > - Possibly postgresql is simply *too big*. Your 50MB(?) Firefox > turns into a 2GB install. I doubt it -- on Debian, postgresql > is 17MB installed. But I don't know what dependencies I'm not > counting. Yeah. And frankly, I would be surprised if Firefox is only 50MB these days. The extra dent of PostgreSQL won't be all that significant - and don't forget that the SQLite3 dent can be removed, so you're talking about the difference between them, plus you can omit a whole bunch of PG's ancillaries. > - Or it is impossible to configure without excessive amounts of > tech-savvy human intervention. Again, I doubt it. I seem to > recall needing to create a Postgresql user and password. But > maybe even that is too technical for the average Firefox user. You don't even need to do that. An absolutely default Postgres on Debian or Ubuntu is ready to use, with peer authentication. If you can become root, you can then drop privs to the 'postgres' user and connect to the database that way. I don't know if it's possible to do a non-root installation of PostgreSQL, but if it isn't today, it could easily be tomorrow, if someone puts in a little effort. You'd miss out on boot-time startup, and it'd probably have to do some password-based authentication (with autogenerated passwords), but it certainly could work. The Windows equivalent ("install for just me") is, I think, already possible. > - Maybe there are nasty interactions between Postgresql listening > on some port and Windows firewall wanting to block that same port. That's definitely an issue, given that Windows doesn't have Unix domain sockets. But I'm sure it's a solvable problem. How does IDLE cope with firewall issues? ChrisA From marfig at gmail.com Thu Feb 19 02:33:11 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 08:33:11 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: On Thu, 19 Feb 2015 18:22:57 +1300, Gregory Ewing wrote: > >How does sqlite3 miss parameterized queries? It supports >DB-API parameter subsitution with '?' according to the >docs. It's actually parameterized views that I meant. Not queries. SQLite misses the ability to write parameterized views. From marfig at gmail.com Thu Feb 19 02:44:15 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 08:44:15 +0100 Subject: python implementation of a new integer encoding algorithm. References: Message-ID: A lot of patronizing egos running around in these groups. This is a sad thread... What is being asked is for help, not whether this is useful or needed. Jan-Hein is after some directions, not whether your bloody opinion on how he should use his free time. If the interest and usability of a project would somehow become a problem, then boy, oh, boy; Most everyone in here, including the patronizing posters, would probably be left without anything to code. From marfig at gmail.com Thu Feb 19 02:49:21 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 08:49:21 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> Message-ID: <0e3beahnqhmn3f3cp9slgq7309v7qh3cev@4ax.com> On Thu, 19 Feb 2015 03:43:36 +0000, Mark Lawrence wrote: >After a wonderful relationship lasting many happy years I dumped Firefox >a few weeks ago for Chrome. A few anxious moments gave me pause for >thought, but overall I'm happy to have changed. However is anybody >aware of a "new kid on the block" that could take over as I'd happily >switch again? Nothing has sprung out at me, hence the choice I made. Despite being built on Chrome framework, Vivaldi seems like an interesting option. Made by the original makers of Opera who have grown annoyed at the direction their browser took. When they opened Vivaldi beta to the public a few weeks ago, it gained good reviews a little all over the web. https://vivaldi.com/ From breamoreboy at yahoo.co.uk Thu Feb 19 03:06:45 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 08:06:45 +0000 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On 19/02/2015 07:44, Mario Figueiredo wrote: > A lot of patronizing egos running around in these groups. This is a > sad thread... > > What is being asked is for help, not whether this is useful or needed. > Jan-Hein is after some directions, not whether your bloody opinion on > how he should use his free time. > > If the interest and usability of a project would somehow become a > problem, then boy, oh, boy; Most everyone in here, including the > patronizing posters, would probably be left without anything to code. > The opinions being expressed seem to be along the lines of "reinventing round wheels is a waste of time. Reinventing square or even triangular wheels is really pointless". If you don't like the opinions expressed here by so called "patronizing posters" you are free to go away at any time you see fit. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ismahameed at gcuf.edu.pk Thu Feb 19 03:35:01 2015 From: ismahameed at gcuf.edu.pk (ismahameed at gcuf.edu.pk) Date: Thu, 19 Feb 2015 00:35:01 -0800 (PST) Subject: urgent help Message-ID: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> this is the error in the following python code, can any one help me error{Traceback (most recent call last): File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in from BeautifulSoup import BeautifulSoup ImportError: No module named BeautifulSoup} "#encoding=utf8 from codecs import open from collections import defaultdict import re from BeautifulSoup import BeautifulSoup import mechanize import cookielib import html2text import time def getbr(): br = mechanize.Browser() # Cookie Jar cj = cookielib.LWPCookieJar() br.set_cookiejar(cj) # Browser options br.set_handle_equiv(True) br.set_handle_gzip(True) br.set_handle_redirect(True) br.set_handle_referer(True) br.set_handle_robots(False) # Follows refresh 0 but not hangs on refresh > 0 br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1) # User-Agent (this is cheating, ok?) br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')] return br def logthis(text): open("log.txt","a","utf8").write(text+"\n") def getCommunity(community,url,out=""): # Browser # The site we will navigate into, handling it's session i = 1 flag = True discussions = [] baseDiscussion = [] while flag: print i currurl = url+"/"+str(i) try: br = getbr() br.open(currurl) #br.follow_link(text='link') html = br.response().read() soup = BeautifulSoup(html) if soup.find("title").string == u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - BeOK\r\n': print "done at ",i,community logthis("done at "+str(i)+" "+community) return True hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"}) print currurl #print hrefList for link in hrefList: #print str(link) #continue span = link.find('div',{"class":"MsgUsr"}) if "frm_mngr" in str(span): mgr = span.find("span",{"class":"frm_mngr"}).string if not "''" in mgr: continue mgr = mgr.replace("'","") date = link.find('span',{"class":"MsgDate"}).string.split(" ")[1] #out.write(community+"\t"+mgr+"\t"+date+"\n") print community.rstrip(),date,mgr #fout = open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8") ansDiv = link.nextSibling.find('div',{"class":"BodyMesInner"}) print "bla" ans = fixHtml2(str(ansDiv)) print "bla" print ans #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n") #fout.close() questionDiv = link.previousSibling.find('div',{"class":"BodyMesInner"}) print "bla",questionDiv quesiton = fixHtml2(str(questionDiv)) print question span = None soup = None br = None except: time.sleep(60) i+=1 return list(set(discussions)) def fixHtml(page): page = page.replace("

","\n") page = page.replace("

","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace(""","'") reg = re.compile("<") reg2 = re.compile(">") page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))]) page = page.replace("\r\n\t\t\t","\n") return page def fixHtml2(page): page = page.split('ner">')[1].split("","\n") page = page.replace("

","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace("
","\n") page = page.replace(""","'") return page def getText(br,url): br.open(url) html = br.response().read() soup = BeautifulSoup(html) title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0]) #print title artics = soup.findAll('div',{'class':"article"}) text = "\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0]) text += "\n"+ fixHtml(str(artics[1]).split('"article">')[1].split('')[0])+"" text = text.decode("utf-8") #text = artics[0] + #print type(title),type(text) return title+text def getForums(file = "links.htm"): #out = open("beokDates","w","utf8") soup = BeautifulSoup(open(file,"r").read()) communities = soup.findAll("a",{"class":"MainList"}) for comm in communities: #print comm["href"] getCommunity(comm.string,comm["href"]) getForums() #links = getQALinks() file = "links.htm" soup = BeautifulSoup(open(file,"r").read()) comm = soup.findAll("a",{"class":"MainList"})[0] br = getbr() currurl = comm["href"]+"/3" br.open(currurl) html = br.response().read() soup = BeautifulSoup(html) hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})[0] " From rosuav at gmail.com Thu Feb 19 03:36:24 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Feb 2015 19:36:24 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 7:06 PM, Mark Lawrence wrote: > On 19/02/2015 07:44, Mario Figueiredo wrote: >> >> A lot of patronizing egos running around in these groups. This is a >> sad thread... >> >> What is being asked is for help, not whether this is useful or needed. >> Jan-Hein is after some directions, not whether your bloody opinion on >> how he should use his free time. > > The opinions being expressed seem to be along the lines of "reinventing > round wheels is a waste of time. Reinventing square or even triangular > wheels is really pointless". If you come to a group of architects and builders and ask them, "What's the best way to build the steps up to my garage?", and show them this picture... http://cheezburger.com/6665198848 ... do you expect advice to switch from concrete to wood, or to scrap the whole idea? ChrisA From marko at pacujo.net Thu Feb 19 03:36:36 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 19 Feb 2015 10:36:36 +0200 Subject: python implementation of a new integer encoding algorithm. References: Message-ID: <878ufub8mj.fsf@elektro.pacujo.net> Mark Lawrence : > The opinions being expressed seem to be along the lines of > "reinventing round wheels is a waste of time. Reinventing square or > even triangular wheels is really pointless". I think it's even more pointless to mention the pointlessness of someone's hobby. Most pointless of all is mentioning the pointlessness of mentioning the pointlessness of someone's hobby... Marko PS On the topic of pointlessness, why is top-posting the norm on python-dev but shunned on python-list? From jonas at wielicki.name Thu Feb 19 03:38:25 2015 From: jonas at wielicki.name (Jonas Wielicki) Date: Thu, 19 Feb 2015 09:38:25 +0100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: <54E5A101.9030207@wielicki.name> Dear Jan-Hein, I read through the discussion, but until you said it directly, I did not realize that you wanted feedback on your *python* code. In that case, let me note a few things which make it unlikely that you will get (usable) feedback: 1. The code on your website is not formatted and highlighted properly. This makes it hard to read. Of course, people can copy-paste it into their favourite highlighter, but that poses a hurdle some (including me) do not want to take, because of ... 2. ... we don?t exactly know how the algorithm is supposed to work. Unfortunately (and I?m quite sure that this is not due to the fact that you?re not native english, because I was able to follow your mails without issues), your description of the algorithm on your blog is full of typos and incomplete or ambiguous grammar. It could also use some mathematical typesetting to make it more readable. I propose that you reformat both your description of the algorithm and your implementation to get a better review on it. In fact, I am quite curious about it (having implemented a MIDI-ish format I stole from Matroska without knowing, for my pet binary storage format, where I need small (less than 8 bit) at a number of occasions), but currently I don?t have the time to dig through it if it doesn?t read fluently (there are exams over here). I assume that many other people also have much other stuff to do. Another idea to make it more attractive for people to review your code (if you are not after a functional review) would be to go to, e.g., . I am not quite sure whether it would be "on-topic" there -- it would probably require a concise description of the algorithm so that people can make their own mind-model about how the algorithm is *supposed* to work and how your code works, to compare and see potential for optimization, code wise. best regards, jwi p.s.: On your website: > The natural character size is 2 bits, and that will eventually become > the standard. What is it with that statement? That bugged me when I visited the page for the first time. Just marketing-wise, I believe you should not confront people with a controversial statement, taking it for granted, on the first glance. It would be better to state that this is your implementation of Algorithm A (link to concise description included) and you would like to get feedback on your implementation or whatever. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From davea at davea.name Thu Feb 19 04:00:50 2015 From: davea at davea.name (Dave Angel) Date: Thu, 19 Feb 2015 04:00:50 -0500 Subject: urgent help In-Reply-To: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> Message-ID: <54E5A642.4090801@davea.name> On 02/19/2015 03:35 AM, ismahameed at gcuf.edu.pk wrote: > this is the error in the following python code, can any one help me > error{Traceback (most recent call last): > File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in > from BeautifulSoup import BeautifulSoup > ImportError: No module named BeautifulSoup} > > > > "#encoding=utf8 > from codecs import open > from collections import defaultdict > import re > > from BeautifulSoup import BeautifulSoup When you can demonstrate a problem in a couple of lines of source code, why would you waste our bandwidth showing us dozens of unrelated lines? Since the error says there's no module named BeautifulSoup, perhaps that's because you haven't installed BeautifulSoup. it's not in the standard library. I've never used it, but a quick web search found me the page: http://www.crummy.com/software/BeautifulSoup/bs4/doc/ And that seems to say the module is called bs4. Anyway, if you did install it, and read the directions, and are still stumped, you probably need to supply many other details: 1) what version of Python are you using, and do you have multiple versions installed 2) what OS 3) where did you download it from, and what commands did you use to actually install it How did you specify which Python version it would install to? 4) what your import line looks like (which you did specify) 5) and of course, what the exception is (which you did include) Other things people may need to know include what directory the bs4.pyc file is installed to, what your sys.path is, and so on. But just answering the first questions might let you figure it out for yourself. -- DaveA From perfectican at gmail.com Thu Feb 19 04:03:28 2015 From: perfectican at gmail.com (perfectican) Date: Thu, 19 Feb 2015 01:03:28 -0800 (PST) Subject: pickle error by multiprocessing In-Reply-To: References: Message-ID: <03097208-b504-49b1-8574-257b8ee34a50@googlegroups.com> I guess, The issue is not caused by multiprocessing. make sure you are pickling right object that can be pickable or not. Best way is run it in a console and test at which object the pickling error is occurring and then try to change the object which could be pickable. From ismahameed at gcuf.edu.pk Thu Feb 19 04:31:20 2015 From: ismahameed at gcuf.edu.pk (ismahameed at gcuf.edu.pk) Date: Thu, 19 Feb 2015 01:31:20 -0800 (PST) Subject: urgent help In-Reply-To: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> Message-ID: <11ec9cc1-0eb4-4981-8a67-2cabf26554f1@googlegroups.com> On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah... at gcuf.edu.pk wrote: > this is the error in the following python code, can any one help me > error{Traceback (most recent call last): > File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in > from BeautifulSoup import BeautifulSoup > ImportError: No module named BeautifulSoup} > > > > "#encoding=utf8 > from codecs import open > from collections import defaultdict > import re > > from BeautifulSoup import BeautifulSoup > import mechanize > import cookielib > import html2text > import time > > > def getbr(): > br = mechanize.Browser() > > # Cookie Jar > cj = cookielib.LWPCookieJar() > br.set_cookiejar(cj) > > # Browser options > br.set_handle_equiv(True) > br.set_handle_gzip(True) > br.set_handle_redirect(True) > br.set_handle_referer(True) > br.set_handle_robots(False) > > # Follows refresh 0 but not hangs on refresh > 0 > br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1) > > # User-Agent (this is cheating, ok?) > br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')] > return br > > def logthis(text): > open("log.txt","a","utf8").write(text+"\n") > > def getCommunity(community,url,out=""): > # Browser > > # The site we will navigate into, handling it's session > i = 1 > > flag = True > discussions = [] > baseDiscussion = [] > > while flag: > print i > currurl = url+"/"+str(i) > try: > br = getbr() > br.open(currurl) > #br.follow_link(text='link') > html = br.response().read() > soup = BeautifulSoup(html) > if soup.find("title").string == u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - BeOK\r\n': > print "done at ",i,community > logthis("done at "+str(i)+" "+community) > return True > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"}) > print currurl > #print hrefList > for link in hrefList: > #print str(link) > #continue > span = link.find('div',{"class":"MsgUsr"}) > > if "frm_mngr" in str(span): > mgr = span.find("span",{"class":"frm_mngr"}).string > if not "''" in mgr: > continue > mgr = mgr.replace("'","") > date = link.find('span',{"class":"MsgDate"}).string.split(" ")[1] > #out.write(community+"\t"+mgr+"\t"+date+"\n") > print community.rstrip(),date,mgr > #fout = open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8") > ansDiv = link.nextSibling.find('div',{"class":"BodyMesInner"}) > print "bla" > ans = fixHtml2(str(ansDiv)) > print "bla" > print ans > #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n") > #fout.close() > questionDiv = link.previousSibling.find('div',{"class":"BodyMesInner"}) > print "bla",questionDiv > quesiton = fixHtml2(str(questionDiv)) > print question > span = None > > > > soup = None > br = None > except: > > time.sleep(60) > i+=1 > return list(set(discussions)) > > def fixHtml(page): > page = page.replace("

","\n") > page = page.replace("

","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace(""","'") > reg = re.compile("<") > reg2 = re.compile(">") > page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))]) > page = page.replace("\r\n\t\t\t","\n") > return page > > def fixHtml2(page): > page = page.split('ner">')[1].split(" print page > page = page.replace("

","\n") > page = page.replace("

","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace("
","\n") > page = page.replace(""","'") > return page > > def getText(br,url): > br.open(url) > html = br.response().read() > soup = BeautifulSoup(html) > title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0]) > #print title > artics = soup.findAll('div',{'class':"article"}) > text = "\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0]) > text += "\n"+ fixHtml(str(artics[1]).split('"article">')[1].split('')[0])+"" > text = text.decode("utf-8") > #text = artics[0] + > #print type(title),type(text) > > return title+text > > def getForums(file = "links.htm"): > #out = open("beokDates","w","utf8") > soup = BeautifulSoup(open(file,"r").read()) > communities = soup.findAll("a",{"class":"MainList"}) > for comm in communities: > #print comm["href"] > getCommunity(comm.string,comm["href"]) > > getForums() > #links = getQALinks() > file = "links.htm" > soup = BeautifulSoup(open(file,"r").read()) > comm = soup.findAll("a",{"class":"MainList"})[0] > br = getbr() > currurl = comm["href"]+"/3" > br.open(currurl) > html = br.response().read() > soup = BeautifulSoup(html) > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})[0] > " yes i have install the beautifulsoup module in python library . From breamoreboy at yahoo.co.uk Thu Feb 19 04:33:19 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 09:33:19 +0000 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <878ufub8mj.fsf@elektro.pacujo.net> References: <878ufub8mj.fsf@elektro.pacujo.net> Message-ID: On 19/02/2015 08:36, Marko Rauhamaa wrote: > > PS On the topic of pointlessness, why is top-posting the norm on > python-dev but shunned on python-list? > I don't know and I don't care provided top-posting remains the norm here. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marfig at gmail.com Thu Feb 19 04:42:51 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 10:42:51 +0100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 9:06 AM, Mark Lawrence wrote > > > The opinions being expressed seem to be along the lines of "reinventing > round wheels is a waste of time. Reinventing square or even triangular > wheels is really pointless". > You obviously don't value the word 'exercise'. > If you don't like the opinions expressed here by so called "patronizing > posters" you are free to go away at any time you see fit. > > If you don't like the opinions on the opinions of patronizing posters, you are free to go away any time you see fit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ismahameed at gcuf.edu.pk Thu Feb 19 04:46:30 2015 From: ismahameed at gcuf.edu.pk (ismahameed at gcuf.edu.pk) Date: Thu, 19 Feb 2015 01:46:30 -0800 (PST) Subject: urgent help In-Reply-To: <11ec9cc1-0eb4-4981-8a67-2cabf26554f1@googlegroups.com> References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> <11ec9cc1-0eb4-4981-8a67-2cabf26554f1@googlegroups.com> Message-ID: <6cce1037-1d2e-4e90-97c8-92ff7f1d677d@googlegroups.com> On Thursday, February 19, 2015 at 5:31:49 PM UTC+8, ismah... at gcuf.edu.pk wrote: > On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah... at gcuf.edu.pk wrote: > > this is the error in the following python code, can any one help me > > error{Traceback (most recent call last): > > File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in > > from BeautifulSoup import BeautifulSoup > > ImportError: No module named BeautifulSoup} > > > > > > > > "#encoding=utf8 > > from codecs import open > > from collections import defaultdict > > import re > > > > from BeautifulSoup import BeautifulSoup > > import mechanize > > import cookielib > > import html2text > > import time > > > > > > def getbr(): > > br = mechanize.Browser() > > > > # Cookie Jar > > cj = cookielib.LWPCookieJar() > > br.set_cookiejar(cj) > > > > # Browser options > > br.set_handle_equiv(True) > > br.set_handle_gzip(True) > > br.set_handle_redirect(True) > > br.set_handle_referer(True) > > br.set_handle_robots(False) > > > > # Follows refresh 0 but not hangs on refresh > 0 > > br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1) > > > > # User-Agent (this is cheating, ok?) > > br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')] > > return br > > > > def logthis(text): > > open("log.txt","a","utf8").write(text+"\n") > > > > def getCommunity(community,url,out=""): > > # Browser > > > > # The site we will navigate into, handling it's session > > i = 1 > > > > flag = True > > discussions = [] > > baseDiscussion = [] > > > > while flag: > > print i > > currurl = url+"/"+str(i) > > try: > > br = getbr() > > br.open(currurl) > > #br.follow_link(text='link') > > html = br.response().read() > > soup = BeautifulSoup(html) > > if soup.find("title").string == u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - BeOK\r\n': > > print "done at ",i,community > > logthis("done at "+str(i)+" "+community) > > return True > > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"}) > > print currurl > > #print hrefList > > for link in hrefList: > > #print str(link) > > #continue > > span = link.find('div',{"class":"MsgUsr"}) > > > > if "frm_mngr" in str(span): > > mgr = span.find("span",{"class":"frm_mngr"}).string > > if not "''" in mgr: > > continue > > mgr = mgr.replace("'","") > > date = link.find('span',{"class":"MsgDate"}).string.split(" ")[1] > > #out.write(community+"\t"+mgr+"\t"+date+"\n") > > print community.rstrip(),date,mgr > > #fout = open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8") > > ansDiv = link.nextSibling.find('div',{"class":"BodyMesInner"}) > > print "bla" > > ans = fixHtml2(str(ansDiv)) > > print "bla" > > print ans > > #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n") > > #fout.close() > > questionDiv = link.previousSibling.find('div',{"class":"BodyMesInner"}) > > print "bla",questionDiv > > quesiton = fixHtml2(str(questionDiv)) > > print question > > span = None > > > > > > > > soup = None > > br = None > > except: > > > > time.sleep(60) > > i+=1 > > return list(set(discussions)) > > > > def fixHtml(page): > > page = page.replace("

","\n") > > page = page.replace("

","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace(""","'") > > reg = re.compile("<") > > reg2 = re.compile(">") > > page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))]) > > page = page.replace("\r\n\t\t\t","\n") > > return page > > > > def fixHtml2(page): > > page = page.split('ner">')[1].split(" > print page > > page = page.replace("

","\n") > > page = page.replace("

","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace("
","\n") > > page = page.replace(""","'") > > return page > > > > def getText(br,url): > > br.open(url) > > html = br.response().read() > > soup = BeautifulSoup(html) > > title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0]) > > #print title > > artics = soup.findAll('div',{'class':"article"}) > > text = "\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0]) > > text += "\n"+ fixHtml(str(artics[1]).split('"article">')[1].split('')[0])+"" > > text = text.decode("utf-8") > > #text = artics[0] + > > #print type(title),type(text) > > > > return title+text > > > > def getForums(file = "links.htm"): > > #out = open("beokDates","w","utf8") > > soup = BeautifulSoup(open(file,"r").read()) > > communities = soup.findAll("a",{"class":"MainList"}) > > for comm in communities: > > #print comm["href"] > > getCommunity(comm.string,comm["href"]) > > > > getForums() > > #links = getQALinks() > > file = "links.htm" > > soup = BeautifulSoup(open(file,"r").read()) > > comm = soup.findAll("a",{"class":"MainList"})[0] > > br = getbr() > > currurl = comm["href"]+"/3" > > br.open(currurl) > > html = br.response().read() > > soup = BeautifulSoup(html) > > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})[0] > > " > > > > > yes i have install the beautifulsoup module in python library . when i checked that the module is working or not then in cmd its show that it is install but when i run my program code then its show that error which i have written before From ismahameed at gcuf.edu.pk Thu Feb 19 04:48:47 2015 From: ismahameed at gcuf.edu.pk (ismahameed at gcuf.edu.pk) Date: Thu, 19 Feb 2015 01:48:47 -0800 (PST) Subject: urgent help In-Reply-To: <6cce1037-1d2e-4e90-97c8-92ff7f1d677d@googlegroups.com> References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> <11ec9cc1-0eb4-4981-8a67-2cabf26554f1@googlegroups.com> <6cce1037-1d2e-4e90-97c8-92ff7f1d677d@googlegroups.com> Message-ID: <9cbcc65d-72ce-423e-9a72-1c81c9c1a7c3@googlegroups.com> On Thursday, February 19, 2015 at 5:46:42 PM UTC+8, ismah... at gcuf.edu.pk wrote: > On Thursday, February 19, 2015 at 5:31:49 PM UTC+8, ismah... at gcuf.edu.pk wrote: > > On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah... at gcuf.edu.pk wrote: > > > this is the error in the following python code, can any one help me > > > error{Traceback (most recent call last): > > > File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in > > > from BeautifulSoup import BeautifulSoup > > > ImportError: No module named BeautifulSoup} > > > > > > > > > > > > "#encoding=utf8 > > > from codecs import open > > > from collections import defaultdict > > > import re > > > > > > from BeautifulSoup import BeautifulSoup > > > import mechanize > > > import cookielib > > > import html2text > > > import time > > > > > > > > > def getbr(): > > > br = mechanize.Browser() > > > > > > # Cookie Jar > > > cj = cookielib.LWPCookieJar() > > > br.set_cookiejar(cj) > > > > > > # Browser options > > > br.set_handle_equiv(True) > > > br.set_handle_gzip(True) > > > br.set_handle_redirect(True) > > > br.set_handle_referer(True) > > > br.set_handle_robots(False) > > > > > > # Follows refresh 0 but not hangs on refresh > 0 > > > br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1) > > > > > > # User-Agent (this is cheating, ok?) > > > br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')] > > > return br > > > > > > def logthis(text): > > > open("log.txt","a","utf8").write(text+"\n") > > > > > > def getCommunity(community,url,out=""): > > > # Browser > > > > > > # The site we will navigate into, handling it's session > > > i = 1 > > > > > > flag = True > > > discussions = [] > > > baseDiscussion = [] > > > > > > while flag: > > > print i > > > currurl = url+"/"+str(i) > > > try: > > > br = getbr() > > > br.open(currurl) > > > #br.follow_link(text='link') > > > html = br.response().read() > > > soup = BeautifulSoup(html) > > > if soup.find("title").string == u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - BeOK\r\n': > > > print "done at ",i,community > > > logthis("done at "+str(i)+" "+community) > > > return True > > > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"}) > > > print currurl > > > #print hrefList > > > for link in hrefList: > > > #print str(link) > > > #continue > > > span = link.find('div',{"class":"MsgUsr"}) > > > > > > if "frm_mngr" in str(span): > > > mgr = span.find("span",{"class":"frm_mngr"}).string > > > if not "''" in mgr: > > > continue > > > mgr = mgr.replace("'","") > > > date = link.find('span',{"class":"MsgDate"}).string.split(" ")[1] > > > #out.write(community+"\t"+mgr+"\t"+date+"\n") > > > print community.rstrip(),date,mgr > > > #fout = open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8") > > > ansDiv = link.nextSibling.find('div',{"class":"BodyMesInner"}) > > > print "bla" > > > ans = fixHtml2(str(ansDiv)) > > > print "bla" > > > print ans > > > #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n") > > > #fout.close() > > > questionDiv = link.previousSibling.find('div',{"class":"BodyMesInner"}) > > > print "bla",questionDiv > > > quesiton = fixHtml2(str(questionDiv)) > > > print question > > > span = None > > > > > > > > > > > > soup = None > > > br = None > > > except: > > > > > > time.sleep(60) > > > i+=1 > > > return list(set(discussions)) > > > > > > def fixHtml(page): > > > page = page.replace("

","\n") > > > page = page.replace("

","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace(""","'") > > > reg = re.compile("<") > > > reg2 = re.compile(">") > > > page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))]) > > > page = page.replace("\r\n\t\t\t","\n") > > > return page > > > > > > def fixHtml2(page): > > > page = page.split('ner">')[1].split(" > > print page > > > page = page.replace("

","\n") > > > page = page.replace("

","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace("
","\n") > > > page = page.replace(""","'") > > > return page > > > > > > def getText(br,url): > > > br.open(url) > > > html = br.response().read() > > > soup = BeautifulSoup(html) > > > title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0]) > > > #print title > > > artics = soup.findAll('div',{'class':"article"}) > > > text = "\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0]) > > > text += "\n"+ fixHtml(str(artics[1]).split('"article">')[1].split('')[0])+"" > > > text = text.decode("utf-8") > > > #text = artics[0] + > > > #print type(title),type(text) > > > > > > return title+text > > > > > > def getForums(file = "links.htm"): > > > #out = open("beokDates","w","utf8") > > > soup = BeautifulSoup(open(file,"r").read()) > > > communities = soup.findAll("a",{"class":"MainList"}) > > > for comm in communities: > > > #print comm["href"] > > > getCommunity(comm.string,comm["href"]) > > > > > > getForums() > > > #links = getQALinks() > > > file = "links.htm" > > > soup = BeautifulSoup(open(file,"r").read()) > > > comm = soup.findAll("a",{"class":"MainList"})[0] > > > br = getbr() > > > currurl = comm["href"]+"/3" > > > br.open(currurl) > > > html = br.response().read() > > > soup = BeautifulSoup(html) > > > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})[0] > > > " > > > > > > > > > > yes i have install the beautifulsoup module in python library . > > when i checked that the module is working or not then in cmd its show that it is install but when i run my program code then its show that error which i have written before i am using windows 8 , and i have installed python27, and i have installed beautifulsoup module from this website http://www.crummy.com/software/BeautifulSoup/bs4 From jacob at blindza.co.za Thu Feb 19 05:07:37 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Thu, 19 Feb 2015 12:07:37 +0200 Subject: Accessible tools In-Reply-To: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> Message-ID: <73FB3C9AE7EE4BCA8A5D0480080C788E@JakesPC> Brian, I am also a fully bind python/PHP developer, and all I really bother with, aside from python interpreter itself is the text editor called edSharp: http://www.empowermentzone.com/edsetup.exe That's been developed by another blind guy who's name you might have heard in other circles, Jamal Mazrui, and it's just a programmers text editor that has a bit of extra functionality built in for use along with screen readers like jaws and NVDA, etc. And, there are also some other general blind programmer mailing lists - I am on both program-l at freelists.org as well as a couple of other topic specific ones, but anyway. Stay well Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." ----- Original Message ----- From: "Bryan Duarte" To: Sent: Wednesday, February 18, 2015 7:43 PM Subject: Accessible tools > Hello all, > > I have been posting to another group which directed me to this group. I am > a blind software engineering student at Arizona State University. I am > currently doing research and have decided to use Python as my developing > language. I am in search of an accessible IDE or other tool set which will > allow me to use the features with a screen reader. I have tried a number > of tools but so far have been unsuccessful in finding one that allows me > to use the tools and features of the IDE with my screen reader. > > I understand that I do not need an IDE to use Python, I am looking for an > IDE to allow me to explore the classes and methods easily and quickly. I > have tried the following tools. > ? pyCharm > ? iPy > ? iPy Notebook > ? Idle > ? and Eclipse > > So far only Eclipse was accessible but I did still incounter some > difficulties like the IDE freezing up when tool tips would pop up. Finally > I am using a Mac mostly but can use Windows, or Linux if need be. Thank > you and I appreciate any and all suggestions. > -- > https://mail.python.org/mailman/listinfo/python-list > From jonas at wielicki.name Thu Feb 19 05:08:26 2015 From: jonas at wielicki.name (Jonas Wielicki) Date: Thu, 19 Feb 2015 11:08:26 +0100 Subject: Accessible tools In-Reply-To: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> Message-ID: <54E5B61A.903@wielicki.name> Dear Bryan, I don?t have a finished solution for you, but I would like to link you to a previous thread on this list: The poster seems to be researching into the direction of developing a speech-friendly IDE. You may want to follow his work. regards, jwi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From breamoreboy at yahoo.co.uk Thu Feb 19 05:28:17 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 10:28:17 +0000 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On 19/02/2015 09:42, Mario Figueiredo wrote: > On Thu, Feb 19, 2015 at 9:06 AM, Mark Lawrence > wrote > > > The opinions being expressed seem to be along the lines of > "reinventing round wheels is a waste of time. Reinventing square or > even triangular wheels is really pointless". > > > You obviously don't value the word 'exercise'. > > > If you don't like the opinions expressed here by so called > "patronizing posters" you are free to go away at any time you see fit. > > > If you don't like the opinions on the opinions of patronizing posters, > you are free to go away any time you see fit. > No thanks as I won't be able to find any more candidates for my dream team. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From nomail at invalid.com Thu Feb 19 05:47:34 2015 From: nomail at invalid.com (ast) Date: Thu, 19 Feb 2015 11:47:34 +0100 Subject: A question about how plot from matplotlib works Message-ID: <54e5bf49$0$3054$426a74cc@news.free.fr> Hello >>> import numpy as np >>> import matplotlib.pyplot as plt >>> x = np.arange(10) >>> y = x**2 >>> x array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> y array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) >>> plt.plot(x,y) [] >>> plt.show() The question is: plt.plot() creates an object "matplotlib.lines.Line2D" but this object is not referenced. So this object should disapear from memory. But this doesn't happens since plt.show() draws the curve on a graphic window. So how does it work ? From breamoreboy at yahoo.co.uk Thu Feb 19 05:49:59 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 10:49:59 +0000 Subject: urgent help In-Reply-To: <9cbcc65d-72ce-423e-9a72-1c81c9c1a7c3@googlegroups.com> References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> <11ec9cc1-0eb4-4981-8a67-2cabf26554f1@googlegroups.com> <6cce1037-1d2e-4e90-97c8-92ff7f1d677d@googlegroups.com> <9cbcc65d-72ce-423e-9a72-1c81c9c1a7c3@googlegroups.com> Message-ID: On 19/02/2015 09:48, ismahameed at gcuf.edu.pk wrote: As Dave Angel said nearly two hours ago the module is called bs4, so the command you need is:- from bs4 import BeautifulSoup In future please don't repeat the entire email just to add a sentence or two, particularly when you do so three times, as some people pay for bandwidth. Thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From perfectican at gmail.com Thu Feb 19 05:59:26 2015 From: perfectican at gmail.com (perfectican) Date: Thu, 19 Feb 2015 02:59:26 -0800 (PST) Subject: What the Pythons docs means by "container" ? In-Reply-To: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: <2b4e5086-cd67-486d-8307-7a0539872cf9@googlegroups.com> in duck-typing, any objects can acts as Container. but only difference is type that we need to consider as a Container or not. From lac at openend.se Thu Feb 19 06:18:38 2015 From: lac at openend.se (Laura Creighton) Date: Thu, 19 Feb 2015 12:18:38 +0100 Subject: can python handle CHIME .spt files (fwd) Message-ID: <201502191118.t1JBIcS2010486@fido.openend.se> So I asked Andrew about PyMol. Here's what he said: ------- Forwarded Message Return-Path: From: Andrew Dalke In-Reply-To: <201502190705.t1J75wx7024389 at fido.openend.se> Date: Thu, 19 Feb 2015 10:55:09 +0100 Message-Id: <47EC31CE-33AC-4426-A874-FF4107F0C9A3 at dalkescientific.com> On Feb 19, 2015, at 8:05 AM, Laura Creighton wrote: > Now discussion has moved to > http://www.pymol.org/ > > Do you know if that will do the job? Or anything else about it? I do not know. My knowledge of that field is rather dated now. The best I can find in a quick search is a RasMol -> PyMol converter in 2,600 lines of Python, at http://sourceforge.net/projects/sbevsl/files/ConSCRIPT/ConSCRIPT-1.0/ConSCRIPT_17Jun10/ https://ritdml.rit.edu/bitstream/handle/1850/9019/SMottarellaAbstract2008.pdf?sequence=8 and quoting from http://blondie.dowling.edu/docman/view.php/24/692/ACA_JUL09.ppt The latest version of ConSCRIPT has coded for almost all of the commands possible in RasMol. While support for some remains minimal, included some that only return incompatibility errors, most commands, including most of those that directly manipulate the image, are supported. Here are some of the commands that have recently been added. I have no experience with it. The review at https://www.mail-archive.com/jmol-users at lists.sourceforge.net/msg16583.html says its works quite well. I don't know how portable it is to the Chime commands. Cheers, Andrew dalke at dalkescientific.com ------- End of Forwarded Message From breamoreboy at yahoo.co.uk Thu Feb 19 06:20:29 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 11:20:29 +0000 Subject: What the Pythons docs means by "container" ? In-Reply-To: <2b4e5086-cd67-486d-8307-7a0539872cf9@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> <2b4e5086-cd67-486d-8307-7a0539872cf9@googlegroups.com> Message-ID: On 19/02/2015 10:59, perfectican wrote: > in duck-typing, any objects can acts as Container. but only difference is type that we need to consider as a Container or not. > This comment makes no sense to me at all. Could you please rephrase it, preferably supplying some context at the same time. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marco.nawijn at colosso.nl Thu Feb 19 07:25:28 2015 From: marco.nawijn at colosso.nl (marco.nawijn at colosso.nl) Date: Thu, 19 Feb 2015 04:25:28 -0800 (PST) Subject: A question about how plot from matplotlib works In-Reply-To: <54e5bf49$0$3054$426a74cc@news.free.fr> References: <54e5bf49$0$3054$426a74cc@news.free.fr> Message-ID: On Thursday, February 19, 2015 at 11:47:53 AM UTC+1, ast wrote: > Hello > > >>> import numpy as np > >>> import matplotlib.pyplot as plt > >>> x = np.arange(10) > >>> y = x**2 > >>> x > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >>> y > array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) > >>> plt.plot(x,y) > [] > >>> plt.show() > > > The question is: > > plt.plot() creates an object "matplotlib.lines.Line2D" but this object is > not referenced. So this object should disapear from memory. But > this doesn't happens since plt.show() draws the curve on a graphic > window. So how does it work ? Hi, I have not checked the source code, but pyplot probably implicitly generates a few objects for you. In particular it probably creates a default figure, so when you say "plt.plot(x,y)", behind the scenes pyplot will request the current figure and add the Line2D items to it. Marco From a24061 at ducksburg.com Thu Feb 19 08:18:57 2015 From: a24061 at ducksburg.com (Adam Funk) Date: Thu, 19 Feb 2015 13:18:57 +0000 Subject: sqlite3 and dates References: Message-ID: <1f4hrbxket.ln2@news.ducksburg.com> On 2015-02-18, Johannes Bauer wrote: > On 18.02.2015 12:21, Chris Angelico wrote: > >> SQLite3 is fine for something that's basically just a more structured >> version of a flat file. You assume that nobody but you has the file >> open, and you manipulate it just the same as if it were a big fat blob >> of JSON, but thanks to SQLite, you don't have to rewrite the whole >> file every time you make a small change. That's fine. But it's the >> wrong tool for any job involving multiple users over a network, and >> quite probably the wrong tool for a lot of other jobs too. > > Your assessment that some tools fit certain problems and don't fit > different problems is entirely correct. SQLite does the job that it is > supposed to do and it fills that nieche well. > >> It's the >> smallest-end piece of software that can truly be called a database. I >> would consider it to be the wrong database for serious accounting >> work, and that's based on the ranting of a majorly-annoyed accountant >> who had to deal with issues in professional systems that had made >> similar choices in back-end selection. > > It probably is the wrong database for serious accounting work, and it's > probably also the wrong database for doing multivariate statistical > analysis on sparse matrices that you store in tables. > > You could similarly argue that a hammer is the wrong tool to drive in a > screw and you'd be correct in that assessment. But it's completely > besides the point. "If your only tool is a hammer, every problem looks like a nail." ;-) -- In the 1970s, people began receiving utility bills for -?999,999,996.32 and it became harder to sustain the myth of the infallible electronic brain. (Verity Stob) From a24061 at ducksburg.com Thu Feb 19 08:21:58 2015 From: a24061 at ducksburg.com (Adam Funk) Date: Thu, 19 Feb 2015 13:21:58 +0000 Subject: sqlite3 and dates References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> Message-ID: On 2015-02-18, Chris Angelico wrote: > On Thu, Feb 19, 2015 at 9:17 AM, wrote: >>> SQLite3 is fine for something that's basically just a more structured >>> version of a flat file. You assume that nobody but you has the file >>> open, and you manipulate it just the same as if it were a big fat blob >>> of JSON, but thanks to SQLite, you don't have to rewrite the whole >>> file every time you make a small change. That's fine. >> >> That's bullshit. Sqlite offers a lot more than that including >> a SQL interface, transactions, referential integrity, constraints >> indexes, triggers and other general relational database features. >> >> That you would equate that to a JSON blob would indicate either >> a profound ignorance about Sqlite or (more likely) a need to >> defend your preference with complete disregard of fact. > > I didn't equate them. I said that SQLite3 is great if you look on it > as an upgrade over a JSON blob. Of course it offers more features than > that, and you don't need to swear at me to make your point. > > But SQLite3 is *not* great if you look on it as a database engine > comparable with DB2, PostgreSQL, and even MySQL. I certainly agree with that bit, but in my own code I can almost never justify the hassle (set-up, security considerations, &c.) of using a database server. TBH, one reason I like SQLite3 is that I can easily move the data file around in the filesystem or between machies. -- "It is the role of librarians to keep government running in difficult times," replied Dramoren. "Librarians are the last line of defence against chaos." (McMullen 2001) From marfig at gmail.com Thu Feb 19 08:27:31 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 14:27:31 +0100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 11:28 AM, Mark Lawrence wrote: > > No thanks as I won't be able to find any more candidates for my dream team. > > I'm glad you like me here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denismfmcmahon at gmail.com Thu Feb 19 09:08:32 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 19 Feb 2015 14:08:32 +0000 (UTC) Subject: urgent help References: <8fa27443-12f7-4ef6-ba6b-4af16abae29d@googlegroups.com> Message-ID: On Thu, 19 Feb 2015 04:00:50 -0500, Dave Angel wrote: > On 02/19/2015 03:35 AM, ismahameed at gcuf.edu.pk wrote: >> this is the error in the following python code, can any one help me >> error{Traceback (most recent call last): >> File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in >> >> from BeautifulSoup import BeautifulSoup >> ImportError: No module named BeautifulSoup} >> >> "#encoding=utf8 from codecs import open from collections import >> defaultdict import re >> >> from BeautifulSoup import BeautifulSoup > When you can demonstrate a problem in a couple of lines of source code, > why would you waste our bandwidth showing us dozens of unrelated lines? > > Since the error says there's no module named BeautifulSoup, perhaps > that's because you haven't installed BeautifulSoup. it's not in the > standard library. > > I've never used it, but a quick web search found me the page: > > http://www.crummy.com/software/BeautifulSoup/bs4/doc/ ********************************************************* ********************************************************* **** **** > **** And that seems to say the module is called bs4. **** **** **** ********************************************************* ********************************************************* It seems that the OP has failed to read your post, the documentation or the examples for the code he is using. As a very strong hint, I have highlighted your fix for his main problem above with a few (ok, several) asterisks. Let's see if he can find it now. If he can't, I don't understand why he bothered to ask for help, because I'm pretty sure you nailed the issue right there, and unless he's going to read the responses to his post to see the answers that are provided it's a bit stupid to post asking for help in the first place. -- Denis McMahon, denismfmcmahon at gmail.com From denismfmcmahon at gmail.com Thu Feb 19 09:16:31 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 19 Feb 2015 14:16:31 +0000 (UTC) Subject: What behavior would you expect? References: Message-ID: On Wed, 18 Feb 2015 21:44:12 -0700, Jason Friedman wrote: > My question is, what would be a reasonable behavior/result/return value > if: > 1. "/path/to/dir" does not exist or is not readable Normally I'd say raise an exception. Whether you choose to use an existing exception (will trying to read a non existent dir raise one anyway?) or define your own is up to you. This condition would probably indicate an error in the data received by the function - you should be given a readable directory. If (and only if) being called with an invalid directory is potentially valid, then you could respond to (1) the same as to (2). > 2. no files match the given pattern Return either None, 0, False or an empty string. In both cases, it is then a matter for the calling code to catch the exception or handle the return value appropriately. -- Denis McMahon, denismfmcmahon at gmail.com From python.list at tim.thechases.com Thu Feb 19 09:17:58 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 19 Feb 2015 08:17:58 -0600 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: <20150219081758.3dbe83ca@bigbox.christie.dr> On 2015-02-19 05:32, Mark Lawrence wrote: > On 19/02/2015 00:08, Mario Figueiredo wrote: > > Parameterized queries is just a pet peeve of mine that I wish to > > include here. SQLite misses it and I miss the fact SQLite misses > > it. The less SQL one needs to write in their code, the happier > > one should be. > > Instead, use the DB-API?s parameter substitution. Put ? as a > placeholder wherever you want to use a value, and then provide a > tuple of values as the second argument to the cursor?s execute() > method. (Other database modules may use a different placeholder, > such as %s or :1.) For example:..." I think Mario was referring to what other back ends call prepared statements. So you do something like sql = "..." # parameters are referenced here conn = sqlite3.connect(...) stmt = conn.prepare(sql) for parameters in list_of_parameters: stmt.execute(*parameters) This saves the SQL processor from recompiling the SQL into internal byte-code every time. It's handy if you know a given query will run multiple times with the same "shape" parameters. It's not essential, and some optimize away the need, but many back-end interfaces support it. -tkc From python.list at tim.thechases.com Thu Feb 19 09:21:48 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 19 Feb 2015 08:21:48 -0600 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <42b9ed87-a40a-4e70-afd3-304dc55fa0f0@googlegroups.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <42b9ed87-a40a-4e70-afd3-304dc55fa0f0@googlegroups.com> Message-ID: <20150219082148.7b23d4dd@bigbox.christie.dr> On 2015-02-18 20:05, rurpy at yahoo.com.dmarc.invalid wrote: > Sqlite offers concurrent access already. > What Sqlite doesn't offer is high performance concurrent write > access. That is, it locks the entire database for the duration > of a write operation. Given that most such operations are pretty > short, for a small number of concurrent writers this is not a > big problem. Though the entire-database-lock, as Steven mentions in another section of this thread, causes issues on network-shared file-systems. -tkc From breamoreboy at yahoo.co.uk Thu Feb 19 10:04:47 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 15:04:47 +0000 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <20150219081758.3dbe83ca@bigbox.christie.dr> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <20150219081758.3dbe83ca@bigbox.christie.dr> Message-ID: On 19/02/2015 14:17, Tim Chase wrote: > On 2015-02-19 05:32, Mark Lawrence wrote: >> On 19/02/2015 00:08, Mario Figueiredo wrote: >>> Parameterized queries is just a pet peeve of mine that I wish to >>> include here. SQLite misses it and I miss the fact SQLite misses >>> it. The less SQL one needs to write in their code, the happier >>> one should be. >> >> Instead, use the DB-API?s parameter substitution. Put ? as a >> placeholder wherever you want to use a value, and then provide a >> tuple of values as the second argument to the cursor?s execute() >> method. (Other database modules may use a different placeholder, >> such as %s or :1.) For example:..." > > I think Mario was referring to what other back ends call prepared > statements. So you do something like > > > sql = "..." # parameters are referenced here > conn = sqlite3.connect(...) > stmt = conn.prepare(sql) > for parameters in list_of_parameters: > stmt.execute(*parameters) > > This saves the SQL processor from recompiling the SQL into internal > byte-code every time. It's handy if you know a given query will run > multiple times with the same "shape" parameters. It's not essential, > and some optimize away the need, but many back-end interfaces support > it. > > -tkc > Is this https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany an equivalent? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Thu Feb 19 10:08:49 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 02:08:49 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 1:16 AM, Denis McMahon wrote: >> 2. no files match the given pattern > > Return either None, 0, False or an empty string. > > In both cases, it is then a matter for the calling code to catch the > exception or handle the return value appropriately. I'd avoid the empty string here, because "absence of file" should be very different from "first file matching pattern". Imagine this naive code: fn = my_function("/path/to/dir", "pattern2") move_to("/path/to/dir/" + fn, "/other/path") In the failure case (nothing matches the pattern), what could happen? Best is it raises an exception, which would come from the my_function line. Next best, it returns None, 0, or False, and you get a TypeError on the deletion, and can trace your way back up. Worst, it returns "" and oops, you just moved the whole directory into the target, instead of just one file. Of course, non-naive code probably prefers a None/0/False return to a raised exception, but in any case, the difference between those two options is fairly small. But I'd avoid the potential confusion with the empty string. Anyway, that's just API bike-shedding :) ChrisA From rosuav at gmail.com Thu Feb 19 10:19:18 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 02:19:18 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <20150219081758.3dbe83ca@bigbox.christie.dr> Message-ID: On Fri, Feb 20, 2015 at 2:04 AM, Mark Lawrence wrote: >> This saves the SQL processor from recompiling the SQL into internal >> byte-code every time. It's handy if you know a given query will run >> multiple times with the same "shape" parameters. It's not essential, >> and some optimize away the need, but many back-end interfaces support >> it. >> >> -tkc >> > > Is this > https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany an > equivalent? That's a different feature, and also useful. Personally, I've never used executemany() for anything other than INSERT statements, though I can imagine using it equally for UPDATE. It's useful only when you have a bulk lot to do all at once; you can't take advantage of it to repeat a common and complex query. Imagine you run a web server that shows some statistical information about your session, on every page; this may require a complex query, which you could retain from one page request to another. But that only matters to performance. ChrisA From bryan0731 at gmail.com Thu Feb 19 10:33:45 2015 From: bryan0731 at gmail.com (Bryan Duarte) Date: Thu, 19 Feb 2015 08:33:45 -0700 Subject: Accessible tools In-Reply-To: <54E5B61A.903@wielicki.name> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> <54E5B61A.903@wielicki.name> Message-ID: <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> Thank you jwi, and Jacob, I took a look at that posting and it seems pretty unique. I am not much interested in the speech driven development, but I am very interested in developing an accessible IDE. A professor and I have been throwing around the idea of developing a completely text based IDE. There are a lot of reasons this could be beneficial to a blind developer and maybe even some sighted developers who are comfortable in the terminal. The idea would be really just to provide a way of easily navigating blocks of code using some kind of tabular formatting, and being able to collapse blocks of code and hearing from a high level information about the code within. All tools and features would obviously be spoken or output in some kind of audio manor. Jacob, I know your name and I do know Jamal's name as well. I think I recall your names from either back in the "Mobile Speak" days, or maybe from the jaws mailing list. Either way thank you for the feedback and I will take a look at edSharp today. The Python interpreter is great for small tests or scripts but for lengthy programs there is no easy way to save your code other than capturing the entire history with extra code and all. How do you typically handle that issue? Thank you both. Oh and before I forget does anyone know how to contact Eric who was developing that accessible speech driven IDE? Thanks > On Feb 19, 2015, at 3:08 AM, Jonas Wielicki wrote: > > Dear Bryan, > > I don?t have a finished solution for you, but I would like to link you > to a previous thread on this list: > > > The poster seems to be researching into the direction of developing a > speech-friendly IDE. You may want to follow his work. > > regards, > jwi > > > -- > https://mail.python.org/mailman/listinfo/python-list From janhein.vanderburg at gmail.com Thu Feb 19 10:45:23 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Thu, 19 Feb 2015 07:45:23 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> Message-ID: <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote: > I'm not necessarily doubting it, just challenging you to provide a data > sample that actually shows it. And of course, I'm not claiming that > 7bit is in any way optimal. You cannot define optimal without first > defining the distribution. Weird results. For a character size 2 the growth processes are shown below. I listed the decimal representations, the difficult representation, a stop bit encoding, and the number of characters they differ in length: 0: 00 00 0 1: 01 01 0 2: 10, 00 10, 00 0 3: 10, 01 10, 01 0 4: 10, 10 11, 00 0 5: 10, 11 11, 01 0 6: 11, 00.00 11, 10, 00 0 7: 11, 00.01 11, 10, 01 0 8: 11, 00.10 11, 11, 00 0 9: 11, 00.11 11, 11, 01 0 10: 11, 01.00 11, 11, 10, 00 1 11: 11, 01.01 11, 11, 10, 01 1 12: 11, 01.10 11, 11, 11, 00 1 13: 11, 01.11 11, 11, 11, 01 1 14: 11, 10.00, 00 11, 11, 11, 10, 00 1 15: 11, 10.00, 01 11, 11, 11, 10, 01 1 16: 11, 10.00, 10 11, 11, 11, 11, 00 1 17: 11, 10.00, 11 11, 11, 11, 11, 01 1 18: 11, 10.01, 00.00 11, 11, 11, 11, 10, 00 1 19: 11, 10.01, 00.01 11, 11, 11, 11, 10, 01 1 20: 11, 10.01, 00.10 11, 11, 11, 11, 11, 00 1 21: 11, 10.01, 00.11 11, 11, 11, 11, 11, 01 1 22: 11, 10.01, 01.00 11, 11, 11, 11, 11, 10, 00 2 23: 11, 10.01, 01.01 11, 11, 11, 11, 11, 10, 01 2 24: 11, 10.01, 01.10 11, 11, 11, 11, 11, 11, 00 2 25: 11, 10.01, 01.11 11, 11, 11, 11, 11, 11, 01 2 26: 11, 10.01, 10.00 11, 11, 11, 11, 11, 11, 10, 00 3 I didn't take the time to prove it mathematically, but these results suggest to me that the complicated encoding beats the stop bit encoding. From janhein.vanderburg at gmail.com Thu Feb 19 10:58:03 2015 From: janhein.vanderburg at gmail.com (janhein.vanderburg at gmail.com) Date: Thu, 19 Feb 2015 07:58:03 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: <02ef99cc-8d3d-46ad-a20a-7aa0aacf17d7@googlegroups.com> On Thursday, February 19, 2015 at 10:52:38 AM UTC+1, Jonas Wielicki wrote: > I read through the discussion, but until you said it directly, I did not > realize that you wanted feedback on your *python* code. Thanks for the tips Jonas. From python.list at tim.thechases.com Thu Feb 19 11:03:54 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 19 Feb 2015 10:03:54 -0600 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <20150219081758.3dbe83ca@bigbox.christie.dr> Message-ID: <20150219100354.6489b11a@bigbox.christie.dr> On 2015-02-19 15:04, Mark Lawrence wrote: > On 19/02/2015 14:17, Tim Chase wrote: >>>> Parameterized queries is just a pet peeve of mine that I wish to >>>> include here. SQLite misses it and I miss the fact SQLite misses >>>> it. The less SQL one needs to write in their code, the happier >>>> one should be. >>> >>> Instead, use the DB-API?s parameter substitution. Put ? as a >>> placeholder wherever you want to use a value, and then provide a >>> tuple of values as the second argument to the cursor?s execute() >>> method. (Other database modules may use a different placeholder, >>> such as %s or :1.) For example:..." >> >> I think Mario was referring to what other back ends call prepared >> statements. > > Is this > https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany > an equivalent? Depends on whether sqlite3.Cursor.executemany() uses https://www.sqlite.org/c3ref/stmt.html under the hood. -tkc From jacob at blindza.co.za Thu Feb 19 11:15:52 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Thu, 19 Feb 2015 18:15:52 +0200 Subject: Accessible tools Message-ID: <20BAA233CEF64355A1230133B7E734B8@JakesPC> I use edSharp as my primary code/text editor, and just save code from there, and while it doesn't offer too many things like design tools, it's pretty much perfect in terms of code block selection, code snippets, code navigation - to a certain extent, even in python, with regards to jumping from block to block, etc. Text based IDE would be nice at times, as long as you could review code nicely enough, etc., and in terms of GUI design I generally just make use of/work with scrollable grids via wxPython, since that cooperates well enough in end result with both sighted, and accessibility API users, etc., but anyway. It just means my interface design is done via a form of layout-by-code, dropping controls in a flexGridSizer as well, but anyway. My one primary thing I would like to be able to handle in terms of development process is better means of working with debugging code execution, etc. - at moment, primarily either print out information to console, during dev process, or at times invoke sound effects/TTS output to keep track, or even, occasionally use pickle to store images of objects for later review, etc., but anyway... Stay well Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." ----- Original Message ----- From: "Bryan Duarte" To: "Jonas Wielicki" Cc: Sent: Thursday, February 19, 2015 5:33 PM Subject: Re: Accessible tools > Thank you jwi, and Jacob, > > I took a look at that posting and it seems pretty unique. I am not much > interested in the speech driven development, but I am very interested in > developing an accessible IDE. A professor and I have been throwing around > the idea of developing a completely text based IDE. There are a lot of > reasons this could be beneficial to a blind developer and maybe even some > sighted developers who are comfortable in the terminal. The idea would be > really just to provide a way of easily navigating blocks of code using > some kind of tabular formatting, and being able to collapse blocks of code > and hearing from a high level information about the code within. All tools > and features would obviously be spoken or output in some kind of audio > manor. > > Jacob, I know your name and I do know Jamal's name as well. I think I > recall your names from either back in the "Mobile Speak" days, or maybe > from the jaws mailing list. Either way thank you for the feedback and I > will take a look at edSharp today. The Python interpreter is great for > small tests or scripts but for lengthy programs there is no easy way to > save your code other than capturing the entire history with extra code and > all. How do you typically handle that issue? Thank you both. > > Oh and before I forget does anyone know how to contact Eric who was > developing that accessible speech driven IDE? Thanks >> On Feb 19, 2015, at 3:08 AM, Jonas Wielicki wrote: >> >> Dear Bryan, >> >> I don?t have a finished solution for you, but I would like to link you >> to a previous thread on this list: >> >> >> The poster seems to be researching into the direction of developing a >> speech-friendly IDE. You may want to follow his work. >> >> regards, >> jwi >> >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list > From nadeesh092 at gmail.com Thu Feb 19 11:18:04 2015 From: nadeesh092 at gmail.com (Nadeesh Dilanga) Date: Thu, 19 Feb 2015 21:48:04 +0530 Subject: About GSOC 2015 Message-ID: Hi, I'm a Computer Science undergraduate student who like to participate in GSOC this year. Do you have any projects willing to publish for gsoc 2015. I am more familiar with Python. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carnovg at gmail.com Thu Feb 19 11:27:36 2015 From: carnovg at gmail.com (Phillip Fleming) Date: Thu, 19 Feb 2015 10:27:36 -0600 Subject: Standard Message-ID: In my opinion, Python will not take off like C/C++ if there is no ANSI standard. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nomail at invalid.com Thu Feb 19 12:50:38 2015 From: nomail at invalid.com (ast) Date: Thu, 19 Feb 2015 18:50:38 +0100 Subject: matplotlib.pyplot documentation Message-ID: <54e62271$0$3010$426a74cc@news.free.fr> hello I got this module documentation with help(matplotlib.pyplot) for each function there is a demo file, for example: barbs(*args, **kw) Plot a 2-D field of barbs. .... **Example:** .. plot:: mpl_examples/pylab_examples/barb_demo.py but I dont find the barb_demo.py on my computer, I made a full search on C:/Python34/ and it's failed Do you know where these demo files are located ? (windows vista, python 3.4) thx From breamoreboy at yahoo.co.uk Thu Feb 19 13:02:10 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 19 Feb 2015 18:02:10 +0000 Subject: matplotlib.pyplot documentation In-Reply-To: <54e62271$0$3010$426a74cc@news.free.fr> References: <54e62271$0$3010$426a74cc@news.free.fr> Message-ID: On 19/02/2015 17:50, ast wrote: > hello > > I got this module documentation with help(matplotlib.pyplot) > > for each function there is a demo file, for example: > > barbs(*args, **kw) > Plot a 2-D field of barbs. > .... > **Example:** > .. plot:: mpl_examples/pylab_examples/barb_demo.py > > but I dont find the barb_demo.py on my computer, I made a full search on > C:/Python34/ and it's failed > > Do you know where these demo files are located ? > (windows vista, python 3.4) > > thx > This http://matplotlib.org/examples/pylab_examples/barb_demo.html was the first hit using a search engine. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Thu Feb 19 13:04:10 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 19 Feb 2015 11:04:10 -0700 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> Message-ID: On Thu, Feb 19, 2015 at 8:45 AM, wrote: > On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote: >> I'm not necessarily doubting it, just challenging you to provide a data >> sample that actually shows it. And of course, I'm not claiming that >> 7bit is in any way optimal. You cannot define optimal without first >> defining the distribution. > > Weird results. > For a character size 2 the growth processes are shown below. > I listed the decimal representations, the difficult representation, a stop bit encoding, and the number of characters they differ in length: > 0: 00 00 0 > 1: 01 01 0 > 2: 10, 00 10, 00 0 > 3: 10, 01 10, 01 0 > 4: 10, 10 11, 00 0 > 5: 10, 11 11, 01 0 > 6: 11, 00.00 11, 10, 00 0 > 7: 11, 00.01 11, 10, 01 0 > 8: 11, 00.10 11, 11, 00 0 > 9: 11, 00.11 11, 11, 01 0 > 10: 11, 01.00 11, 11, 10, 00 1 > 11: 11, 01.01 11, 11, 10, 01 1 > 12: 11, 01.10 11, 11, 11, 00 1 > 13: 11, 01.11 11, 11, 11, 01 1 > 14: 11, 10.00, 00 11, 11, 11, 10, 00 1 > 15: 11, 10.00, 01 11, 11, 11, 10, 01 1 > 16: 11, 10.00, 10 11, 11, 11, 11, 00 1 > 17: 11, 10.00, 11 11, 11, 11, 11, 01 1 > 18: 11, 10.01, 00.00 11, 11, 11, 11, 10, 00 1 > 19: 11, 10.01, 00.01 11, 11, 11, 11, 10, 01 1 > 20: 11, 10.01, 00.10 11, 11, 11, 11, 11, 00 1 > 21: 11, 10.01, 00.11 11, 11, 11, 11, 11, 01 1 > 22: 11, 10.01, 01.00 11, 11, 11, 11, 11, 10, 00 2 > 23: 11, 10.01, 01.01 11, 11, 11, 11, 11, 10, 01 2 > 24: 11, 10.01, 01.10 11, 11, 11, 11, 11, 11, 00 2 > 25: 11, 10.01, 01.11 11, 11, 11, 11, 11, 11, 01 2 > 26: 11, 10.01, 10.00 11, 11, 11, 11, 11, 11, 10, 00 3 > > I didn't take the time to prove it mathematically, but these results suggest to me that the complicated encoding beats the stop bit encoding. That stop-bit variant looks extremely inefficient (and wrong) to me. First, 2 bits per group is probably a bad choice for a stop-bit encoding. It saves some space for very small integers, but it won't scale well at all. Fully half of the bits are stop bits! Secondly, I don't understand why the leading groups are all 11s and only the later groups introduce variability. In fact, that's practically a unary encoding with just a small amount of binary at the end. This is what I would expect a 2-bit stop-bit encoding to look like: 0: 00 1: 01 2: 11, 00 3: 11, 01 4: 11, 10, 00 5: 11, 10, 01 6: 11, 11, 00 7: 11, 11, 01 8: 11, 10, 10, 00 9: 11, 10, 10, 01 10: 11, 10, 11, 00 11: 11, 10, 11, 01 12: 11, 11, 10, 00 13: 11, 11, 10, 01 14: 11, 11, 11, 00 15: 11, 11, 11, 01 16: 11, 10, 10, 10, 00 17: 11, 10, 10, 10, 01 18: 11, 10, 10, 11, 00 19: 11, 10, 10, 11, 01 20: 11, 10, 11, 10, 00 21: 11, 10, 11, 10, 01 22: 11, 10, 11, 11, 00 23: 11, 10, 11, 11, 01 24: 11, 11, 10, 10, 00 25: 11, 11, 10, 10, 01 26: 11, 11, 10, 11, 00 27: 11, 11, 10, 11, 01 28: 11, 11, 11, 10, 00 29: 11, 11, 11, 10, 01 30: 11, 11, 11, 11, 00 31: 11, 11, 11, 11, 01 etc. Notice that the size grows as O(log n), not O(n) as above. Notice also that the only values here for which this saves space over the 7-bit version are 0-7. Unless you expect those values to be very common, the 7-bit encoding that needs only one byte all the way up to 127 makes a lot of sense. There's also an optimization that can be added here if we wish to inject a bit of cleverness. Notice that all values with more than one group start with 11, never 10. We can borrow a trick from IEEE floating point and make the leading 1 bit of the mantissa implicit for all values greater than 3 (we can't do it for 2 and 3 because then we couldn't distinguish them from 0 and 1). Applying this optimization removes one full group from the representation of all values greater than 3, which appears to make the stop-bit representation as short as or shorter than the "difficult" one for all the values that have been enumerated above. From ian.g.kelly at gmail.com Thu Feb 19 13:16:10 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 19 Feb 2015 11:16:10 -0700 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> Message-ID: On Thu, Feb 19, 2015 at 11:04 AM, Ian Kelly wrote: > There's also an optimization that can be added here if we wish to > inject a bit of cleverness. Notice that all values with more than one > group start with 11, never 10. We can borrow a trick from IEEE > floating point and make the leading 1 bit of the mantissa implicit for > all values greater than 3 (we can't do it for 2 and 3 because then we > couldn't distinguish them from 0 and 1). Applying this optimization > removes one full group from the representation of all values greater > than 3, which appears to make the stop-bit representation as short as > or shorter than the "difficult" one for all the values that have been > enumerated above. I made a mistake. This trick only works if the second group is 10, so it only applies to half the numbers. That makes 12 and 13 a group longer than the difficult representation, but on the other hand 18-23 are still a group shorter. From lynto28 at gmail.com Thu Feb 19 13:16:10 2015 From: lynto28 at gmail.com (Didymus) Date: Thu, 19 Feb 2015 10:16:10 -0800 (PST) Subject: Logging with Custom Levels not working In-Reply-To: References: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> Message-ID: <674667d9-a720-4d3f-979b-3ea2c2267e8b@googlegroups.com> On Wednesday, February 18, 2015 at 3:16:40 PM UTC-5, Ian wrote: > > def perror(self, message, *args, **kws): > > """ Performance Error Message Level """ > > # Yes, logger takes its '*args' as 'args'. > > self._log(PERROR_NUM, message, args, **kws) > > > > logging.Logger.perror = perror > > I think you need to call self.log, not self._log. The _log method > appears to assume that the level check has already been performed. You > really shouldn't be calling it directly anyway, as the leading _ is an > indication that the method is not part of the public API. Yes, I had tried that and get: Logged from file log5.py, line 21 Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 851, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 464, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting I also added: ll = rootLogger.getEffectiveLevel() which send back the correct level, but it's still printing out everything... -Tom From davea at davea.name Thu Feb 19 13:24:40 2015 From: davea at davea.name (Dave Angel) Date: Thu, 19 Feb 2015 13:24:40 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> Message-ID: <54E62A68.4010603@davea.name> On 02/19/2015 10:45 AM, janhein.vanderburg at gmail.com wrote: > On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote: >> I'm not necessarily doubting it, just challenging you to provide a data >> sample that actually shows it. And of course, I'm not claiming that >> 7bit is in any way optimal. You cannot define optimal without first >> defining the distribution. > > Weird results. In what way weird? > For a character size 2 the growth processes are shown below. > I listed the decimal representations, the difficult representation, a stop bit encoding, and the number of characters they differ in length: > 0: 00 00 0 > 1: 01 01 0 > 2: 10, 00 10, 00 0 > 3: 10, 01 10, 01 0 > 4: 10, 10 11, 00 0 > 5: 10, 11 11, 01 0 > 6: 11, 00.00 11, 10, 00 0 > 7: 11, 00.01 11, 10, 01 0 > 8: 11, 00.10 11, 11, 00 0 > 9: 11, 00.11 11, 11, 01 0 > 10: 11, 01.00 11, 11, 10, 00 1 > 11: 11, 01.01 11, 11, 10, 01 1 > 12: 11, 01.10 11, 11, 11, 00 1 > 13: 11, 01.11 11, 11, 11, 01 1 > 14: 11, 10.00, 00 11, 11, 11, 10, 00 1 > 15: 11, 10.00, 01 11, 11, 11, 10, 01 1 > 16: 11, 10.00, 10 11, 11, 11, 11, 00 1 > 17: 11, 10.00, 11 11, 11, 11, 11, 01 1 > 18: 11, 10.01, 00.00 11, 11, 11, 11, 10, 00 1 > 19: 11, 10.01, 00.01 11, 11, 11, 11, 10, 01 1 > 20: 11, 10.01, 00.10 11, 11, 11, 11, 11, 00 1 > 21: 11, 10.01, 00.11 11, 11, 11, 11, 11, 01 1 > 22: 11, 10.01, 01.00 11, 11, 11, 11, 11, 10, 00 2 > 23: 11, 10.01, 01.01 11, 11, 11, 11, 11, 10, 01 2 > 24: 11, 10.01, 01.10 11, 11, 11, 11, 11, 11, 00 2 > 25: 11, 10.01, 01.11 11, 11, 11, 11, 11, 11, 01 2 > 26: 11, 10.01, 10.00 11, 11, 11, 11, 11, 11, 10, 00 3 > > I didn't take the time to prove it mathematically, but these results suggest to me that the complicated encoding beats the stop bit encoding. > Clearly, one wouldn't use what you call "stop bit encoding" with a 2bit character. And since the Python code you posted uses an 8bit character, I can't validate the "difficult" column either. I wrote the following pair of functions: def seven_code(n): acc = bytearray() if n == 0: acc.append(0) while n > 0: quotient, remainder = divmod(n, 128) acc.append(remainder) n = quotient acc[-1] |= 0x80 #add a stop bit to last byte return acc def seven_decode(sequ): acc = 0 multiplier = 1 for by in sequ: acc += (by & 0x7f) * multiplier if by > 0x7f: break multiplier *= 128 return acc Here's a couple of ranges of output, showing that the 7bit scheme does better for values between 384 and 16379. 382 2 80fe --- 2 7e82 383 2 80ff --- 2 7f82 384 3 810000 --- 2 0083 384 jan grew 3 810000 385 3 810001 --- 2 0183 386 3 810002 --- 2 0283 387 3 810003 --- 2 0383 388 3 810004 --- 2 0483 389 3 810005 --- 2 0583 16380 3 813e7c --- 2 7cff 16380 jan grew 3 813e7c 16380 7bit grew 2 7cff 16381 3 813e7d --- 2 7dff 16382 3 813e7e --- 2 7eff 16383 3 813e7f --- 2 7fff 16384 3 813e80 --- 3 000081 16384 7bit grew 3 000081 16385 3 813e81 --- 3 010081 16386 3 813e82 --- 3 020081 16387 3 813e83 --- 3 030081 16388 3 813e84 --- 3 040081 16389 3 813e85 --- 3 050081 In all my experimenting, I haven't found any values where the 7bit scheme does worse. It seems likely that for extremely large integers, it will, but if those are to be the intended distribution, the 7bit scheme could be replaced by something else, like just encoding a length at the beginning, and using raw bytes after that. -- DaveA From ian.g.kelly at gmail.com Thu Feb 19 13:32:17 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 19 Feb 2015 11:32:17 -0700 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E62A68.4010603@davea.name> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> Message-ID: On Thu, Feb 19, 2015 at 11:24 AM, Dave Angel wrote: > Here's a couple of ranges of output, showing that the 7bit scheme does > better for values between 384 and 16379. > > 382 2 80fe --- 2 7e82 > 383 2 80ff --- 2 7f82 > 384 3 810000 --- 2 0083 > 384 jan grew 3 810000 > 385 3 810001 --- 2 0183 > 386 3 810002 --- 2 0283 > 387 3 810003 --- 2 0383 > 388 3 810004 --- 2 0483 > 389 3 810005 --- 2 0583 > > 16380 3 813e7c --- 2 7cff > 16380 jan grew 3 813e7c > 16380 7bit grew 2 7cff > 16381 3 813e7d --- 2 7dff > 16382 3 813e7e --- 2 7eff > 16383 3 813e7f --- 2 7fff > 16384 3 813e80 --- 3 000081 > 16384 7bit grew 3 000081 > 16385 3 813e81 --- 3 010081 > 16386 3 813e82 --- 3 020081 > 16387 3 813e83 --- 3 030081 > 16388 3 813e84 --- 3 040081 > 16389 3 813e85 --- 3 050081 > > In all my experimenting, I haven't found any values where the 7bit scheme > does worse. It seems likely that for extremely large integers, it will, but > if those are to be the intended distribution, the 7bit scheme could be > replaced by something else, like just encoding a length at the beginning, > and using raw bytes after that. It looks like you're counting whole bytes, not bits. That would be important since the "difficult" encoding uses fractional bytes. From rosuav at gmail.com Thu Feb 19 13:34:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 05:34:58 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E62A68.4010603@davea.name> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> Message-ID: On Fri, Feb 20, 2015 at 5:24 AM, Dave Angel wrote: > In all my experimenting, I haven't found any values where the 7bit scheme > does worse. It seems likely that for extremely large integers, it will, but > if those are to be the intended distribution, the 7bit scheme could be > replaced by something else, like just encoding a length at the beginning, > and using raw bytes after that. Encoding a length (as varlen) and then using eight bits to the byte thereafter is worse for small numbers, breaks even around 2**56, and then is better. So unless your numbers are mainly going to be above 2**56, it's better to just use varlen for the entire number. On the other hand, if you have to stream this without over-reading (imagine streaming from a TCP/IP socket; you want to block until you have the whole number, but not block after that), it may be more efficient to take the length, and then do a blocking read for the main data, instead of a large number of single-byte reads. But on the gripping hand, you can probably just do those one-byte reads and rely on (or implement) lower-level buffering. Ask not the python-list for advice, because they will say both "yes" and "no" and "maybe"... because they will say all three of "yes", "no", "maybe", and "you don't need to do that"... erm, AMONG our responses will be such diverse elements as... ChrisA From davea at davea.name Thu Feb 19 13:41:48 2015 From: davea at davea.name (Dave Angel) Date: Thu, 19 Feb 2015 13:41:48 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> Message-ID: <54E62E6C.4030508@davea.name> On 02/19/2015 01:34 PM, Chris Angelico wrote: > On Fri, Feb 20, 2015 at 5:24 AM, Dave Angel wrote: >> In all my experimenting, I haven't found any values where the 7bit scheme >> does worse. It seems likely that for extremely large integers, it will, but >> if those are to be the intended distribution, the 7bit scheme could be >> replaced by something else, like just encoding a length at the beginning, >> and using raw bytes after that. > > Encoding a length (as varlen) and then using eight bits to the byte > thereafter is worse for small numbers, I only suggested this if it turns out that the distribution is primarily extremely large numbers, large enough that 7bit isn't good enough. As I (and others) have said many times, making it optimal means making some assumptions about the distribution of likely values. > breaks even around 2**56, and > then is better. So unless your numbers are mainly going to be above > 2**56, it's better to just use varlen for the entire number. On the > other hand, if you have to stream this without over-reading (imagine > streaming from a TCP/IP socket; you want to block until you have the > whole number, but not block after that), it may be more efficient to > take the length, and then do a blocking read for the main data, > instead of a large number of single-byte reads. But on the gripping > hand, you can probably just do those one-byte reads and rely on (or > implement) lower-level buffering. > > Ask not the python-list for advice, because they will say both "yes" > and "no" and "maybe"... because they will say all three of "yes", > "no", "maybe", and "you don't need to do that"... erm, AMONG our > responses will be such diverse elements as... > > ChrisA > -- DaveA From python.list at tim.thechases.com Thu Feb 19 13:43:46 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 19 Feb 2015 12:43:46 -0600 Subject: Accessible tools In-Reply-To: <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> <54E5B61A.903@wielicki.name> <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> Message-ID: <20150219124346.7a7db5c5@bigbox.christie.dr> While not blind, I have an interest in accessibility and answer a number of questions on the Blinux (Blind Linux Users) mailing list. On 2015-02-19 08:33, Bryan Duarte wrote: > A professor and I have been throwing around the idea of developing > a completely text based IDE. There are a lot of reasons this could > be beneficial to a blind developer and maybe even some sighted > developers who are comfortable in the terminal. The idea would be > really just to provide a way of easily navigating blocks of code > using some kind of tabular formatting, and being able to collapse > blocks of code and hearing from a high level information about the > code within. All tools and features would obviously be spoken or > output in some kind of audio manor. It would seem that the traditional Unix-as-IDE[1] would serve you well here. This is my method of choice, and it allows me to pick my components and combine them. I usually use tmux, though GNU screen would do too. Within that, I usually have the following: - vim to edit my code. Though swap in your favorite, whether emacs/emacspeak, ed/edbrowse, joe, nano, or whatever. I know that at least Vim and emacs support "folding" away blocks of code (what you describe as "collapsing") which I usually prefix with a comment that would give you a description of the block - a command-line (I use bash, some prefer zsh or tcsh or whatever) for things like version-control, running my code, and file management (move/copy/delete/rename/link/etc) - a Python command-line REPL that allows me to do quick tests on a line of code as well as well as make extensive use of Python's built-in dir() and help() commands which are invaluable. - when doing web-development (Django in my case), I'll often have the dev-server running in one pane, and a console browser like lynx/links/links2/elinks/w3m in another pane so that I can put my code through its paces Another benefit of this is that I can run this on my development machine, but then SSH into the machine from anywhere, reattach to the tmux/screen session, and have the same configuration right as I left it. The entire tmux/screen session can be run within an accessible terminal window (I know that some are more accessible than others), within a terminal screen-reader session (like yasr, screader, or emacspeak), or even remoted into via an accessible SSH program on your platform of choice. -tkc [1] http://blog.sanctum.geek.nz/series/unix-as-ide/ From davea at davea.name Thu Feb 19 13:46:23 2015 From: davea at davea.name (Dave Angel) Date: Thu, 19 Feb 2015 13:46:23 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> Message-ID: <54E62F7F.9050504@davea.name> On 02/19/2015 01:32 PM, Ian Kelly wrote: > On Thu, Feb 19, 2015 at 11:24 AM, Dave Angel wrote: >> Here's a couple of ranges of output, showing that the 7bit scheme does >> better for values between 384 and 16379. >> >> 382 2 80fe --- 2 7e82 >> 383 2 80ff --- 2 7f82 >> 384 3 810000 --- 2 0083 >> 384 jan grew 3 810000 >> 385 3 810001 --- 2 0183 >> 386 3 810002 --- 2 0283 >> 387 3 810003 --- 2 0383 >> 388 3 810004 --- 2 0483 >> 389 3 810005 --- 2 0583 >> >> 16380 3 813e7c --- 2 7cff >> 16380 jan grew 3 813e7c >> 16380 7bit grew 2 7cff >> 16381 3 813e7d --- 2 7dff >> 16382 3 813e7e --- 2 7eff >> 16383 3 813e7f --- 2 7fff >> 16384 3 813e80 --- 3 000081 >> 16384 7bit grew 3 000081 >> 16385 3 813e81 --- 3 010081 >> 16386 3 813e82 --- 3 020081 >> 16387 3 813e83 --- 3 030081 >> 16388 3 813e84 --- 3 040081 >> 16389 3 813e85 --- 3 050081 >> >> In all my experimenting, I haven't found any values where the 7bit scheme >> does worse. It seems likely that for extremely large integers, it will, but >> if those are to be the intended distribution, the 7bit scheme could be >> replaced by something else, like just encoding a length at the beginning, >> and using raw bytes after that. > > It looks like you're counting whole bytes, not bits. That would be > important since the "difficult" encoding uses fractional bytes. > Not the implementation that was shared. I've only seen one set of Python code for "difficult", and it was strictly bytes. As i said earlier in the message you quoted from. Naturally, I question whether the original description makes sense for sub-bytes, since it was claimed that these are NOT for lists or sequences of arbitrary integers, but only for a single one at a time. Presumably mixed with other data which may or may not like bit encoding. -- DaveA From rosuav at gmail.com Thu Feb 19 13:49:27 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 05:49:27 +1100 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <54E62E6C.4030508@davea.name> References: <515047c1-a20d-430e-a029-1c2d77db2f1a@googlegroups.com> <2a717ffb-d61d-4407-9082-1c17cd7ee573@googlegroups.com> <993f64c3-fc85-48a7-9d02-a4f12ecb33c6@googlegroups.com> <54E62A68.4010603@davea.name> <54E62E6C.4030508@davea.name> Message-ID: On Fri, Feb 20, 2015 at 5:41 AM, Dave Angel wrote: > As I (and others) have said many times, making it optimal means making some > assumptions about the distribution of likely values. In fact, the very word "optimal" implies that. You have to have a set of criteria on which you base your evaluation, and those criteria will include target data. ChrisA From rurpy at yahoo.com Thu Feb 19 14:45:32 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 11:45:32 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <20150219081758.3dbe83ca@bigbox.christie.dr> Message-ID: On 02/19/2015 09:03 AM, Tim Chase wrote: > On 2015-02-19 15:04, Mark Lawrence wrote: >> On 19/02/2015 14:17, Tim Chase wrote: >>>>> Parameterized queries is just a pet peeve of mine that I wish to >>>>> include here. SQLite misses it and I miss the fact SQLite misses >>>>> it. The less SQL one needs to write in their code, the happier >>>>> one should be. >>>> >>>> Instead, use the DB-API's parameter substitution. Put ? as a >>>> placeholder wherever you want to use a value, and then provide a >>>> tuple of values as the second argument to the cursor's execute() >>>> method. (Other database modules may use a different placeholder, >>>> such as %s or :1.) For example:..." >>> >>> I think Mario was referring to what other back ends call prepared >>> statements. >> >> Is this >> https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany >> an equivalent? > > Depends on whether sqlite3.Cursor.executemany() uses > > https://www.sqlite.org/c3ref/stmt.html > > under the hood. So it seems that Sqlite does have prepared statements -- they are just accessible from the api and not from SQL. Regarding Mark's question I would say that a more significant difference is that executemany() requires you to know all the bind parameter values at the time the statement is executed. A prepared statement does not require you to know any of the bind parameter values when you prepare the statement -- they are supplied to the prepared statement at any later time or times when you execute the prepared statement. From tjreedy at udel.edu Thu Feb 19 14:50:24 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 19 Feb 2015 14:50:24 -0500 Subject: python implementation of a new integer encoding algorithm. In-Reply-To: <878ufub8mj.fsf@elektro.pacujo.net> References: <878ufub8mj.fsf@elektro.pacujo.net> Message-ID: On 2/19/2015 3:36 AM, Marko Rauhamaa wrote: > PS On the topic of pointlessness, why is top-posting the norm on > python-dev ... ? It isn't, except that Guido gets a special pass and some of the posters travel a lot and read and reply on phones, which makes snipping and inline response difficult, and so they top post even though it makes their responsed *much* harder to understand. -- Terry Jan Reedy From marko at pacujo.net Thu Feb 19 14:55:06 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 19 Feb 2015 21:55:06 +0200 Subject: python implementation of a new integer encoding algorithm. References: <878ufub8mj.fsf@elektro.pacujo.net> Message-ID: <87oaopn0bp.fsf@elektro.pacujo.net> Terry Reedy : > It isn't, except that Guido gets a special pass Wusses... Or, it's good to be king. Marko From denismfmcmahon at gmail.com Thu Feb 19 15:03:38 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Thu, 19 Feb 2015 20:03:38 +0000 (UTC) Subject: What behavior would you expect? References: Message-ID: On Fri, 20 Feb 2015 02:08:49 +1100, Chris Angelico wrote: > On Fri, Feb 20, 2015 at 1:16 AM, Denis McMahon > > wrote: >>> 2. no files match the given pattern >> >> Return either None, 0, False or an empty string. >> >> In both cases, it is then a matter for the calling code to catch the >> exception or handle the return value appropriately. > > I'd avoid the empty string here, because "absence of file" should be > very different from "first file matching pattern". Imagine this naive > code: If your function documentation states that a failure to match any existing file returns an empty string (and if that's the case, then the documentation should do), then whoever calls the function should check the return value isn't an empty string. There's two conflicting "paradigms" as it were here. On the one hand, the return type of a function (when it returns, rather than raising an exception) should be consistent to itself, even if using a language where types are not declared. On the other hand, a failure condition should clearly be a failure condition, which for a language that supports untyped functions means that we have the luxury of being able to return None / Nul[l] / NaN / False / 0 etc instead of a string / object / array / list / dictionary / mapping etc instead of raising an exception for the failure, and so we can try and handle the failure at the calling level without worrying about trapping exceptions. I guess at the end of the day the programmer has to consider and determine which is most appropriate to his application, given the case. As an aside, it's always a good idea to check that what you get looks like what you expected, whether it comes from a function call or as a data packet over the internet, before you start using it to do other things. ;) -- Denis McMahon, denismfmcmahon at gmail.com From rosuav at gmail.com Thu Feb 19 15:11:13 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 07:11:13 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 7:03 AM, Denis McMahon wrote: > On the one hand, the return type of a function (when it returns, rather > than raising an exception) should be consistent to itself, even if using > a language where types are not declared. > Yes, so I'd advise against having a function sometimes return a string and sometimes a dict. Sure. But None gets a bit of a special pass here; in a lot of languages, what you'd have is a "nullable" type (eg in C you might declare the return value as "pointer to char", and either return a pointer to a string, or NULL), but in Python, you return None for the special case. So in a sense, "str or None" is not so inconsistent. > I guess at the end of the day the programmer has to consider and > determine which is most appropriate to his application, given the case. Indeed. But knowing that your caller quite possibly won't check, what would you do, to increase the chances of the unexpected being noticed? Again, it comes down to expectations. If 25% of queries are going to return "nothing found", then having that be the empty string is fine - you can be pretty sure your users will test for it. But if that's an extremely rare case, then it may be worth raising an exception instead, so it's safe even if someone forgets to test for the unusual. ChrisA From rurpy at yahoo.com Thu Feb 19 15:20:03 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 12:20:03 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <54e58ba8$0$11121$c3e8da3@news.astraweb.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On 02/19/2015 12:07 AM, Steven D'Aprano wrote: > rurpy at yahoo.com wrote: >> On 02/18/2015 07:13 PM, Steven D'Aprano wrote: >>> Chris Angelico wrote: >>>>> SQLite misses some important features that makes it better suited as a >>>>> simple datastore, not much unlike shelve. And network use is not one >>>>> of them, since you can actually implement concurrent sqlite access by >>>>> coding an intermediate layer. Assuming of course we are talking about >>>>> a small number of concurrent users. >>>> >>>> This is what I was saying: it's fine for purposes like Firefox's >>>> bookmarks and settings and such (which I think was what it was >>>> originally developed for?). Not so fine over a network. >>> >>> The sheer number of Firefox bugs related to its use of SQLite says >>> different. >>> >>> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain >>> text files. At worst they were HTML. You could trivially read them, copy >>> them, restore them and even (if you were careful) edit them using the >>> text editor of your choice. Many a time I was on one machine, wanted to >>> know a bookmark from another machine, so I would ssh across to the other >>> machine and run grep over the bookmark file. >> >> I agree, I prefer plain text files whenever practical. But since >> the original discussion was about Sqlite vs Postgresql, not Sqlite >> vs text files, shouldn't the question be: would Firefox be better >> if it required you to install and configure Postgreql instead of >> using Sqlite? > > Very possibly. With modern dependency management, it isn't hard to install > Postgresql: > > sudo aptitude postgresql > > or equivalent should work. And the equivalent for Android would be? Even with Windows, Mac and Linux I don't imagine they want to be in the business of tracking what package managers are used by what OSes and OS version and dealing with the inevitable problems that will arise, particularly with a userbase that is 99.9% non- technical. Compare with Sqlite which require zero end-user involvement and puts the management in one place, in-house, and under control of the developers. > For primitive operating systems with no > dependency management available, Firefox could come with a simple script > which downloads, installs, configures and runs Postgresql. (Everything is > simple for the guy who doesn't have to do it.) Right. The key is in the last sentence above. > Possible snags: > [...] > > - Or it is impossible to configure without excessive amounts of > tech-savvy human intervention. Again, I doubt it. I seem to > recall needing to create a Postgresql user and password. But > maybe even that is too technical for the average Firefox user. You want hands-off for non-technical users and hands-on for technical ones. The last thing I want is yet another database server running -- if I want FF to use Postgresql, I want it to use a database in my existing server for which I already have management and backup/recovery procedures established. So now you have to ask me if I want to use an existing server or not and if so what its connection details are. And this is not per-install but per profile creation. My point is not that this is an insoluble problem -- just that it is one of many such problems that take some non-trivial amount of time recognize and to address -- time which is not necessary when using Sqlite. > - Maybe there are nasty interactions between Postgresql listening > on some port and Windows firewall wanting to block that same port. The attack surface for any application that has a network port is vastly greater that one that uses only filesystem apis like sqlite. So you've now just undertaken to be responsible to a much greater degree for the security of my machine. FF does already have a bucketful of security issues but as a client, not as a server. And what about backing up all that data in the wonderful, featureful database you just installed for me? As you know, you can't just include the postresql data files in a file system backup. And what about upgrades to the server? Are you going to develop your own upgrade infrastructure, or are you going to tell me to use the standard Postgresql upgrade methodology (which involves, in simple cases, dumping the data with pg_dumpall, doing the upgrade, and restoring with pg_restore_all). If the latter, I'm sure that will be really popular with non-technical users. Of course with sqlite, you just ship a new shared library file with your new version of Firefox. > Or... and here is a radical thought... maybe Firefox could give you the > choice of which database? By default, it might use Sqlite, to satisfy the > desktop users who don't want to think about it. And for those who are > disturbed by the fragility of Sqlite on a network home directory, you just > set a config setting in about:config to point at your existing Postgresql > instance, and never need worry about it again. Yes, I personally would like that a lot. But I'm not going to stay up at night waiting for it since the number of people like me and you are a minuscule fraction of FFs userbase. There is next to no incentive for the Mozilla devs to include such an esoteric feature that benefits so few users and has a high cost (compared to Sqlite) to add and maintain. > The Firefox devs surprise and confuse me. On the one hand, they have > designed a powerful plug-in architecture, and encourage their user-base to > use it for all sorts of amazing functionality that they don't want to build > into the core browser. Yay for this. And on the other hand, they are > *actively hostile* to any suggestion that using SQlite is not the best and > *only* appropriate solution to the problem of storing config, bookmarks and > history. A plug-in database architecture would probably work really well. > >> I don't see any evidence that it is Sqlite that is the problem >> as opposed to FF's use (or misuse) of it, or other problems that >> are in FF and have nothing to do with Sqlite. > > No no, even Sqlite devs recommend against using it on network drives. Yes, you are correct. I realized that after I posted but I decided getting some sleep was more important than correcting myself. > The > Firefox problem is that when FF crashes, as it can do, or if you yank the > power to the computer and everything dies, if your home directory is on a > network drive, the database may be left in a locked state, or even > corrupted. Nothing that the FF developers can do, given the choice of > Sqlite. > >> If Sqlite reliably >> implements ACID semantics as they claim, > > Ah, well "reliably" is a tricky word... > > http://stackoverflow.com/questions/788517/sqlite-over-a-network-share Thanks for that reference. Now that I know that someone named "anon" out on the internet thinks that file-based database on shared drives are unreliable, I will definitely add that to my "facts" draw. :-) Seriously, I don't dispute the basic claim but how about some credible specifics rather than FUD? Since you don't supply any I will make an attempt: To reliably implement the D (durable) in ACID, the database has to be able to know when data has been permanently written to disk. I don't know if there are wan shared file systems whose protocols include such notification but I suspect not many to none do. Thus, placing data on a shared filesystem is a problem for any database, this is not Sqlite specific. Try putting your Postgresql data directory on a remote NFS share and let us know how it works out for you. So again, I don't see how this is specifically Sqlites's fault. I can see your point questioning FF's use of Sqlite but the idea of replacing it with Postgresql I think is just downright nutty. From rowen at uw.edu Thu Feb 19 15:20:40 2015 From: rowen at uw.edu (Russell Owen) Date: Thu, 19 Feb 2015 12:20:40 -0800 Subject: How to install PIL or PILLOW on OS X Yosemite? In-Reply-To: References: Message-ID: On 2/15/15 8:17 PM, Ned Deily wrote: > In article , > KP wrote: >> just upgraded my Mac Mini to Yosemite and have never dabbled in Python on >> this OS. >> >> I see it has Python 2.7.6 installed. >> >> When I do something like >> >> from PIL import ImageFont, ImageDraw >> >> it tells me that it cannot find PIL >> >> How do I install this on Yosemite? > > Suggestions: stick with Pillow which is the current, maintained fork of > the venerable PIL. Decide whether you want to use Python 3 or Python 2. > PIL/Pillow installation on OS X is more involved than on some other > platforms because it depends on a number of third-party C libraries that > are not shipped by Apple in OS X so you need to find another source for > them. Rather than trying to build and install everything yourself or > downloading a Pillow or PIL installer, I suggest picking one of the > several fine distributors of open source packages for OS X and > installing everything you need from them (including an up-to-date Python > 2 or 3) and for your future needs beyond Pillow; options include > Homebrew, MacPorts, Anaconda, Fink, and others. Once you've installed > the base framework for the package manager you choose, installing > something like Pillow and all of its dependencies is often just a > one-line command. It may take a little while to get used to the quirks > of the package manager you choose but, if you are going to use OS X for > development with Python or many other languages, that time spent will be > repaid many times over. I agree that Pillow is preferable to PIL and that you may want to consider a 3rd party system. If you are primarily interested in Python (and not unix-based C/C++ libraries and utilities then I suggest you try anaconda python. Homebrew, MacPorts and Fink are mostly aimed at people who want to add missing unix libraries and tools. If you want to stick with python.org python then a binary PIL installer is available here: (I am not aware of any Pillow binaries). -- Russell From rurpy at yahoo.com Thu Feb 19 15:26:04 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 12:26:04 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On 02/19/2015 12:23 AM, Chris Angelico wrote: > On Thu, Feb 19, 2015 at 6:07 PM, Steven D'Aprano wrote: >> Very possibly. With modern dependency management, it isn't hard to install >> Postgresql: >> >> sudo aptitude postgresql >> >> or equivalent should work. For primitive operating systems with no >> dependency management available, Firefox could come with a simple script >> which downloads, installs, configures and runs Postgresql. (Everything is >> simple for the guy who doesn't have to do it.) > > Definitely a possibility. I'm pretty sure I've seen that exact thing > done by one application (on Windows; on a typical Linux system, > that'll be done by simple dependency management - your package > metadata says "depends on postgresql" and apt-get or yum or whatever > will do the work), and it's quite common for a Windows installer > wizard to go through a series of subcomponents (grab the .NET > framework, grab these hotfixes that the program depends on, grab some > adware toolbar that you forgot to untick, and *then* install the > program you asked for). > >> Possible snags: >> >> - Possibly postgresql is simply *too big*. Your 50MB(?) Firefox >> turns into a 2GB install. I doubt it -- on Debian, postgresql >> is 17MB installed. But I don't know what dependencies I'm not >> counting. > > Yeah. And frankly, I would be surprised if Firefox is only 50MB these > days. The extra dent of PostgreSQL won't be all that significant - and > don't forget that the SQLite3 dent can be removed, so you're talking > about the difference between them, plus you can omit a whole bunch of > PG's ancillaries. > >> - Or it is impossible to configure without excessive amounts of >> tech-savvy human intervention. Again, I doubt it. I seem to >> recall needing to create a Postgresql user and password. But >> maybe even that is too technical for the average Firefox user. > > You don't even need to do that. An absolutely default Postgres on > Debian or Ubuntu is ready to use, with peer authentication. If you can > become root, you can then drop privs to the 'postgres' user and > connect to the database that way. > > I don't know if it's possible to do a non-root installation of > PostgreSQL, but if it isn't today, it could easily be tomorrow, if > someone puts in a little effort. You'd miss out on boot-time startup, > and it'd probably have to do some password-based authentication (with > autogenerated passwords), but it certainly could work. The Windows > equivalent ("install for just me") is, I think, already possible. > >> - Maybe there are nasty interactions between Postgresql listening >> on some port and Windows firewall wanting to block that same port. > > That's definitely an issue, given that Windows doesn't have Unix > domain sockets. But I'm sure it's a solvable problem. How does IDLE > cope with firewall issues? There's an awful lot of "could"s, "possible"s etc in your and Steven's responses. And most of those words apply to issues that aren't problems at all when using Sqlite. I'll point out that five people in this thread (by my count) have said that Postgresql requires a significant amount of work to setup and use. Only you and Steven claim the opposite. (And it sounds to me like Steven does not have a lot of experience installing, configuring and maintaining Postgresql -- please correct me if I am wrong Steven.) I've used Postgresql for a number of small to medium size projects for work and personal use. I too think it is an amazing piece of work for free software. Consequently I used it for a small educational app I developed for my personal use. A number of friends saw it, liked it and asked about using it themselves. Unfortunately none of these friends were at all technical and the barrier of installing and configuring Postresql would have involved me in trying to talk them through it on the phone (risky as there were OSs I was unfamiliar with involved) or do it myself in person (one of them was 2000mi away) or do it remotely (again a cost in coming up with and installing remote desktop software on different OSs and non-technical doubts about its use ("I don't mind you doing this while I'm siting beside you but remote access is a different story"). Then there are the ongoing maintenance issues I mentioned in my last reply to Steven. None of these would have been impossible to overcome with more effort and expense. Or I could have tried as you and Steven suggest to "somehow" package Postgresql in my app installer. That would have been an even bigger cost in my time with an uncertain outcome. None of those costs would have been necessary at all had I developed a self-contained Sqlite app. Finally keep in mind that if you develop your app using Sqlite, it is likely to be far easier to migrate to a heavy-duty backend like Postgresql later should you need to than to go in the other direction when you find out you didn't really need Postgresql after all and the cost turned out to be higher than you expected. Postresql is a wonderful tool when used where its features are needed. It is not so wonderful used when a less heavy- weight solution is can provide the necessary functionality. To insist on using Postgresql in such circumstances is technical masturbation. It is bad advise to recommend using Postgresql without regard to the developer's actual needs. From rosuav at gmail.com Thu Feb 19 15:47:59 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 07:47:59 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On Fri, Feb 20, 2015 at 7:26 AM, wrote: > I'll point out that five people in this thread (by my > count) have said that Postgresql requires a significant > amount of work to setup and use. Only you and Steven claim > the opposite. (And it sounds to me like Steven does not > have a lot of experience installing, configuring and > maintaining Postgresql -- please correct me if I am wrong > Steven.) I don't know about Steven, but I do have a certain amount of experience installing, configuring, and maintaining; and the biggest thing I've noted about the "configuring" part is that you can ignore it most of the time. The docs caution you that the defaults are deliberately NOT set for maximum performance... but you won't normally need maximum performance initially anyway. (In any case, "maximum performance" requires a lot of knowledge of workloads, resource availability, concurrency model, etc.) The defaults are usually good enough; the most common changes I've needed to make to Postgres configs are: 1) Weakening the default security model of listening only on localhost, to allow other computers on the network to connect to the database. Obviously it's correct for the default to be secure, here; anyone who wants to allow remote access will need to be aware of what s/he is doing anyway, so requiring that a couple of files be edited isn't a big deal. 2) Increasing logging, eg turning on the logging of all queries. Not particularly common, but can be handy; and it's a trivial edit anyway. 3) Permitting certain special-purpose users to bypass the normal login mechanisms, eg for scripted backups. Similarly to #1, the defaults are absolutely correct, and if you want to weaken your security, it's only fitting that you have to go in and edit some configs. Given that I frequently don't need to do _any_ of these, it's safe to say that configuring PostgreSQL doesn't take a lot of effort at all. Similarly, "maintaining" isn't really a thing; apart from maintaining my own data tables (cleaning out junk tables that were for one quick test, which I often forget to drop when I'm done with them), I don't have to spend any effort making sure the database keeps running. (In contrast, a busy MySQL server probably needs to have myisamchk run on it periodically, which *does* count as maintenance.) So while it makes sense to talk about *experience* "installing, configuring, and maintaining", it's really just the installing part that actually demands effort. And that can be mitigated in a few ways, but ultimately, there's no single simple solution for everything. Even SQLite3 has issues with that - as soon as you have two programs that try to read the same file (eg your application, using Python's module, and the sqlite3 command line tool), you have version differences, and quite a number of my students have come to me asking why on earth they can't upgrade their SQLite3 properly. (Though usually both versions are "new enough", so it doesn't matter. It's just a point of confusion.) ChrisA From jason.swails at gmail.com Thu Feb 19 16:05:19 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 19 Feb 2015 16:05:19 -0500 Subject: A question about how plot from matplotlib works In-Reply-To: <54e5bf49$0$3054$426a74cc@news.free.fr> References: <54e5bf49$0$3054$426a74cc@news.free.fr> Message-ID: On Thu, Feb 19, 2015 at 5:47 AM, ast wrote: > Hello > > import numpy as np >>>> import matplotlib.pyplot as plt >>>> x = np.arange(10) >>>> y = x**2 >>>> x >>>> >>> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >> y >>>> >>> array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) > >> plt.plot(x,y) >>>> >>> [] > >> plt.show() >>>> >>> > > The question is: > > plt.plot() creates an object "matplotlib.lines.Line2D" but this object is > not referenced. So this object should disapear from memory. But > this doesn't happens since plt.show() draws the curve on a graphic > window. So how does it work ? ?A reference to it is put in the "active" Axes instance of the matplotlib.pyplot namespace. There are many things that will prevent an object from being garbage-collected (a common source of references are caches). [1] ?In general, matplotlib has many containers. In particular, Line2D objects generated by the "plot" function are added to the Axes instance from which "plot" was called. When you don't explicitly specify an Axes object from which to plot, matplotlib.pyplot applies it to some "default" Axes instance living in the matplotlib.pyplot namespace.? This is done to give matplotlib more of a Matlab-like feel. To demonstrate this, let's go try and FIND that reference to the lines: >>> import matplotlib.pyplot as plt ?>>> import numpy as np ?>>> x = np.arange(10) ?>>> y = x ** 2 ?>>> x ?array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) ?>>> y ?array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) ?>>> lines, = plt.plot(x, y) ?>>> id(lines) ?4466622800 ?? >>> lines ? ?>>> del lines ?>>> # Now let's find those lines ?... active_axes = plt.gca() # Get Current Axes ?>>> dir(active_axes) ?[..., get_lines, ...] <-- this is snipped for brevity ?>>> active_axes.get_lines() ? ?>>> active_axes.get_lines()[0] ? ?>>> id(active_axes.get_lines()[0]) ?4466622800 And there we have it! Success! (Note, my comment indicates that the gca in plt.gca() stands for "Get Current Axes"). I also snipped the list of attributes in active_axes that I got from the "dir" command, since that list is HUGE, but the method we want is, rather expectedly, "get_lines". In *my* personal opinion, the matplotlib API is quite intuitive, such that, coupled with Python's native introspective functions (like dir() and id()) and "help" function in the interpreter, I rarely have to consult StackOverflow or even the API documentation online to do what I need. For instance, you want to change the color or thickness of the error bar hats on error bars in your plot? Either save a reference to them when they are generated (by plt.errorbar, for instance), or go *find* them inside the Axes you are manipulating and set whatever properties you want. Hope this helps, Jason [1] OK, so there are not many *things* -- only if there are active, non-circular references will the object *not* be garbage-collected, loosely speaking. But there are many reasons and places that such references are generated inside many APIs... caching being one of the most popular. -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher -------------- next part -------------- An HTML attachment was scrubbed... URL: From joepureddu at yahoo.com Thu Feb 19 16:08:19 2015 From: joepureddu at yahoo.com (DAVIDE SERRA IN GALERA, PLS.) Date: Thu, 19 Feb 2015 13:08:19 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From rurpy at yahoo.com Thu Feb 19 16:17:53 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 13:17:53 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <85lhjuahsm.fsf@benfinney.id.au> <858ufua6ej.fsf@benfinney.id.au> <854mqia56m.fsf@benfinney.id.au> <85wq3e8or6.fsf@benfinney.id.au> Message-ID: On Wednesday, February 18, 2015 at 10:39:04 PM UTC-7, Ethan Furman wrote: > On 02/18/2015 09:26 PM, memilanuk wrote: > > On 02/18/2015 09:16 PM, Ben Finney wrote: > >> memilanuk writes: > >> > >>> In the past I've been waffling back and forth between a desktop > >>> client/server setup, or a web-based interface with everything on one > >>> computer. At this point I'm leaning toward the latter. > >> > >> So, it's been many exchanges back and forth, and you still aren't > >> telling us what specific needs you have that SQLite can't provide. At > >> this point I'm just going to have to wait until you can lay out the > >> specifics. > >> > > > > Okay, let me put it like this: if I set up a web interface using Flask for the front-end, and SQLite as the backend DB, > > running from a PC/laptop, with anywhere from 1 to 10 people doing data entry from other devices (laptops, desktops, > > tablets, etc.) at roughly the same time, is SQLite going to be 'concurrent' enough? > > Well, having zero experience with SQLite, but having read the docs just today [snip snide remark] -- I think you'll be > fine with SQLite under those conditions. :) I too agree with this (with a similar caveat -- I've only used Sqlite for playing around and have no real experience using it in real-world applications). Another thing to keep in mind is if you do need find you need a more heavy duty backend, it will be easier to migrate from Sqlite to Postgresql (or whatever) [*] than to go the other way should you later find Postgresl too heavy-weight and costly. ---- [*] Unless you are very careful to use only features in Postgresql that you've determined can be easily migrated back to Sqlite. In practice very few people have the strength of character required to do this. :-) From marfig at gmail.com Thu Feb 19 16:23:47 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Thu, 19 Feb 2015 22:23:47 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On Thu, 19 Feb 2015 12:26:04 -0800 (PST), rurpy at yahoo.com wrote: > >I'll point out that five people in this thread (by my >count) have said that Postgresql requires a significant >amount of work to setup and use. Only you and Steven claim >the opposite. Well, I claim the opposite too. >Or I could have tried as you >and Steven suggest to "somehow" package Postgresql in my >app installer. That would have been an even bigger cost >in my time with an uncertain outcome. I don't see how. You said it was just a simple application this postgres database was serving. So I must assume you aren't talking of a complex setup that database synchronization or other features that force a manual setup. For your purposes, you could just basically backup your data folder and package postgres with the same configuration files you have in your current computer. >None of those >costs would have been necessary at all had I developed >a self-contained Sqlite app. The cost would have probably been much higher, depending on your project. SQLite would have forced you to move all your business logic into your code, greatly increasing code maintenance, your application extensibility and its ability to more easily adapt to new business requirements. Conversely, if none of this is true concerning your particular project, then you just chose the wrong tool. Postgres was overkill for your particular needs and it was a mistake to think you need it to function just as a shelve on steroids. > >Finally keep in mind that if you develop your app using >Sqlite, it is likely to be far easier to migrate to >a heavy-duty backend like Postgresql later should you >need to than to go in the other direction when you find >out you didn't really need Postgresql after all and the >cost turned out to be higher than you expected. Completely not true! For the reasons mentioned above. You are concentrating too much on the RDBMS aspects and completely forgetting about the implications in your codebase. Whether you move from a non distributed model to a client-server model, or the other way around, your code will suffer major changes. And even if you decide to keep the business logic in the client layer (which is a mistake) when moving from SQLite to a client-server RDBMS, you will still have to deal with a whole new set of issues regarding the very nature of concurrent access that will essentially force you to scrap much of your previous code. >It is bad advise to recommend using Postgresql without >regard to the developer's actual needs. Naturally. But I must say postgres isn't the bad cat you painted in your post. It's much, much easier to distribute, deploy and manage than you are suggesting. For most systems where performance and database synchronization aren't a requirement, it can be entirely automated, I know, because that's how we had it set up on three schools where we sold our integrated management system. I haven't had a maintenance request call in 8 months. It's only under critical requirements that postgres necessitates a baby sitter. And those projects don't suffer from lack of competent administrators. From jt11378 at gmail.com Thu Feb 19 21:42:30 2015 From: jt11378 at gmail.com (Jay T) Date: Thu, 19 Feb 2015 18:42:30 -0800 (PST) Subject: Python - parsing nested information and provide it in proper format from log file Message-ID: <0097dab0-301c-42e1-a6be-b21eb5356567@googlegroups.com> have some log file which has nested data which i want to filter and provide specific for student with total counts Here is my log file sample: Student name is ABC Student age is 12 student was late student was late student was late Student name is DEF student age is 13 student was late student was late i want to parse and show data as Student name, student age , number of counts how many times student was late e:g Name Age TotalCount ABC 12 3 DEF 13 2 Please help me with solution that will be really grateful. thanks, Jt From denismfmcmahon at gmail.com Thu Feb 19 21:46:25 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Fri, 20 Feb 2015 02:46:25 +0000 (UTC) Subject: python implementation of a new integer encoding algorithm. References: Message-ID: On Tue, 17 Feb 2015 03:22:47 -0800, janhein.vanderburg wrote: > In http://optarbvalintenc.blogspot.nl/ I propose a new way to encode > arbitrarily valued integers I'm not quite sure I understand the problem that you're trying to solve with this. If I want to transmit some arbitrarily huge integer value between two systems there are several ways to do it: Numeric text - roughly 24 bits are used to carry 10 bits of value Alphanumeric text - 8 bits are used to carry every 4 bits of value A bit sequence of length n bits where 2^n > the value I wish to convey, providing that there is some mechanism by which the sender can tell the receiver "the next n bits represent a binary integer." It might be desirable to pad the bit stream to a byte boundary. Assuming an 8 bit clean transmission path and a maximum padding of 7 bits, this is increasingly efficient as the integer being transmitted gets larger. The thing is, for all practical purposes, any integer that is capable of being processed as an integer by a computer is probably already being stored in a binary format in a storage space of n bytes, where n is a power of 2. Very little additional processing should be needed to packetize those n bytes and transmit them. Additionally, you're talking about efficiency and the need to optimise, but you're using a scripted language. If you need a highly efficient protocol to transfer binary numbers between two systems with minimum wasted bits and maximum cpu and memory efficiency, python really isn't the language in which to solve your problem. Perhaps it's time to take a step back and redefine the problem a bit more clearly, because at the moment I'm not sure you're solution will ever solve anything that needs solving. -- Denis McMahon, denismfmcmahon at gmail.com From rurpy at yahoo.com Thu Feb 19 23:20:42 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 20:20:42 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: <4d7a8fd2-144a-4400-b012-1967f6deeded@googlegroups.com> On 02/19/2015 01:47 PM, Chris Angelico wrote: > On Fri, Feb 20, 2015 at 7:26 AM, wrote: >> I'll point out that five people in this thread (by my >> count) have said that Postgresql requires a significant >> amount of work to setup and use. Only you and Steven claim >> the opposite. (And it sounds to me like Steven does not >> have a lot of experience installing, configuring and >> maintaining Postgresql -- please correct me if I am wrong >> Steven.) > > I don't know about Steven, but I do have a certain amount of > experience installing, configuring, and maintaining; Right. I am aware of that as you might have guessed by the fact that I directed my question to Steven and not to you. :-) > [...] the most common changes I've needed to make to Postgres configs are: >[...] > 2) Increasing logging, eg turning on the logging of all queries. Not > particularly common, but can be handy; and it's a trivial edit anyway. It may be trivial to you but it is not going to be trivial to many of end users the kind of apps we're talking about. One of the people interested in my little app was at the limits of her ability when it came to finding a PowerPoint file she'd just been editing so she could attach it to an email. So if the logging level needs changing, the app better be prepared to do it itself. >[...] > So while it makes sense to talk about *experience* "installing, > configuring, and maintaining", it's really just the installing part > that actually demands effort. No, see below. > And that can be mitigated in a few ways, > but ultimately, there's no single simple solution for everything. That's right but misses the obvious: some solutions are simpler than others. Installing a Postgresql client-server database and a Python application is inherently more complex than installing a Python application (Sqlite being included in Python). (Doesn't that seem rather obvious, even to you?) And, if using Postgresql is as easy as you claim, surely you don't expect to reserve its use only to your applications? So we could expect to see many applications, potentially all those currently using Sqlite, to start using Postgresql. Do they all install and run their own independent server? Each with its own set of more than a half dozen processes running all the time? Each that has to be started up at system boot? Each with their own directory of Postgresql software and backend data directory? Where do these files all go and how do app developers agree on some standard to avoid chaos? How does each app find a network port or socket that doesn't conflict with the others or with things the machine owner may run? For a Sqlite app you are installing files for which the rules are all pretty well established. And if the Postresql apps share a server who arbitrates when app X decides it needs to set some parameter that breaks app Y? Or change anything else on the server; things shared in common by all the apps like users or (as you yourself mentioned) log file settings? The complexities you deny exist do exist and have to be dealt with somewhere. Either you, the developer have to handle them in the installation and anticipate problems like "Install error: /var/lib/pgsql/data: already exists" or "data directory contain wrong version of Postresql" or you shove the task off onto your users in which case they will have to have sufficient expertise to deal with them. Install problems are not of course limited to server software, they occur with simple Python apps too. But installing a single Python app that depends on the version of Sqlite in the minimally supported version of Python from a self-contained installer will have far lower probability of problems than an automated install of server software in a very uncontrolled environment. And whatever that app install failure probability is, requiring server software in addition can only increase it. > Even > SQLite3 has issues with that - as soon as you have two programs that > try to read the same file (eg your application, using Python's module, > and the sqlite3 command line tool), you have version differences, and > quite a number of my students have come to me asking why on earth they > can't upgrade their SQLite3 properly. (Though usually both versions > are "new enough", so it doesn't matter. It's just a point of > confusion.) Irrelevant. As the application designer you are responsible for what you provide (the application and its backend database). If I want to obtain a command line tool somewhere, it's my problem to worry about versions -- it has nothing to do with you. If you mean that you as a developer have to worry about not shipping two incompatible versions of app code, well... yeah. I think that's pretty much a standard part of putting together a working application. Or is there some other point I missed? As for no maintenance, I mentioned two common maintenance activities in my reply to Steven: backups and server upgrades. From rurpy at yahoo.com Thu Feb 19 23:27:48 2015 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Thu, 19 Feb 2015 20:27:48 -0800 (PST) Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <54E517B4.4000409@stoneleaf.us> <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> Message-ID: On 02/19/2015 02:23 PM, Mario Figueiredo wrote: > On Thu, 19 Feb 2015 12:26:04 -0800 (PST), rurpy at yahoo.com wrote: >> >> I'll point out that five people in this thread (by my >> count) have said that Postgresql requires a significant >> amount of work to setup and use. Only you and Steven claim >> the opposite. > > Well, I claim the opposite too. > >> Or I could have tried as you >> and Steven suggest to "somehow" package Postgresql in my >> app installer. That would have been an even bigger cost >> in my time with an uncertain outcome. > > I don't see how. You said it was just a simple application this > postgres database was serving. So I must assume you aren't talking of > a complex setup that database synchronization or other features that > force a manual setup. Correct. > For your purposes, you could just basically backup your data folder > and package postgres with the same configuration files you have in > your current computer. What do you mean by "data folder"? The directory in which Postgresql keeps its database data? If so you're wrong. That data is at a minimum architecture dependent. It is also private to Postgresql and one would be ill advised to use that as a distribution format. There is also no need to since installing Postgresql and loading the initial data from some standard format is supported and will produce the same results. Or are you talking about some sort of data (csv files, sql files of insert statements, etc) that you use to initially load tables in your database. That will be the same whether you are using Postgresql or Sqlite so I don't see your point. >> None of those >> costs would have been necessary at all had I developed >> a self-contained Sqlite app. > > The cost would have probably been much higher, depending on your > project. SQLite would have forced you to move all your business logic > into your code, greatly increasing code maintenance, your application > extensibility and its ability to more easily adapt to new business > requirements. How so? nobody's claimed that Sqlite is a replacement for Postgresql in large scale, high concurrency "heavy-duty" applications. Ethan Furman posted a list of things that Sqlite does well from the Sqlite website. Here is a link if you missed it: http://www.sqlite.org/whentouse.html We are talking about using Postgresql as a backend for applications that fit on that list, ie applications like the applications already using Sqlite :-) >From previous posts I am guessing that what you are saying is that business logic should be implemented in stored procedures and because Sqlite does not offer stored procedures you can't implement business logic in Sqlite. First, recall that we are not talking about a multi-tiered set of applications with a middleware layer and mutiuser backend. We've established already that is the domain of servers like Postgresql. While I tend to agree with the idea that business logic should be in the database, I'm sure you're aware that that is not a universally held opinion and it is certainly not a universally implemented one in the domain of applications we're talking about. It is easy to find plenty of applications that implement all or part of the business logic in the app. Given Sqlite's architecture I'm not even sure you can say it doesn't have stored procedures. Since programmatic access is through the (python-)API you can write a module with all the business logic and decree that that all higher level functions access the database through that module. How is that effectively different than a set of stored procedures in a client-server database? And I can see a justification for not even going that far in some cases. We are talking about small scale applications where the database can be considered an "implementation detail" of the application but it is easier to implement the application by taking advantage of the capabilities of a SQL relational database than trying to implement those storage details in some ad-hoc way. > Conversely, if none of this is true concerning your particular > project, then you just chose the wrong tool. Postgres was overkill for > your particular needs and it was a mistake to think you need it to > function just as a shelve on steroids. Bingo! You're catching on. :-) Again I remind you that no one has said that Sqlite is a universal replacement for Postgresql >> Finally keep in mind that if you develop your app using >> Sqlite, it is likely to be far easier to migrate to >> a heavy-duty backend like Postgresql later should you >> need to than to go in the other direction when you find >> out you didn't really need Postgresql after all and the >> cost turned out to be higher than you expected. > > Completely not true! For the reasons mentioned above. You are > concentrating too much on the RDBMS aspects and completely forgetting > about the implications in your codebase. > > Whether you move from a non distributed model to a client-server > model, or the other way around, your code will suffer major changes. > And even if you decide to keep the business logic in the client layer > (which is a mistake) Whether it is a mistake or not is not does not have a simple yes/no answer. See above. > when moving from SQLite to a client-server RDBMS, > you will still have to deal with a whole new set of issues regarding > the very nature of concurrent access that will essentially force you > to scrap much of your previous code. How so? We've already determined that Sqlite provides concurrent access (albeit on a limited scale). So you've already had to deal with it in the way you've designed the database. I think you are saying that if you are moving to Postresql that you will want to restructure your code to take advantage of Postgreql's greater capabilities. Sure, but your exiting existing Sqlite design will still work while you do that restructure incrementally. The code will need changes to adapt from Sqlite to Postgresql but that should be tractable. The converse (moving from Postgreql back to Sqlite is much harder because now you have reimplement those nice features of Postgreql that you took advantage of when writing your application but that are hard to port to Sqlite. So your app is broken until you do all that work. >> It is bad advise to recommend using Postgresql without >> regard to the developer's actual needs. > > Naturally. But I must say postgres isn't the bad cat you painted in > your post. It's much, much easier to distribute, deploy and manage > than you are suggesting. If you think I've been painting Postgresql as a "bad cat" I don't think you've been paying attention. On 02/19/2015 01:26 PM, rurpy wrote: > I've used Postgresql for a number of small to medium size > projects for work and personal use. I too think it is an > amazing piece of work for free software. What I have said is that there are many use cases for a SQL relational database that don't need the features of Postgresql and for which Sqlite is a lighter-weight, less costly alternative and hence the better choice. Saying that something is not appropriate for use in every possible environment is not the same as saying it's a "bad cat". > For most systems where performance and database synchronization aren't > a requirement, it can be entirely automated, I know, because that's > how we had it set up on three schools where we sold our integrated > management system. I haven't had a maintenance request call in 8 > months. I never said it can't be done. I claim it can't be done anywhere as near as easily as you can with Sqlite *if you don't require the capabilities of Postgresql*. How long did it take to develop your automated installer and over how wide a variety of environments does it work? Three sites, all institutional, is not what I'd call a a wide variety. They all have someone (perhaps defacto) in the role of systems or network administrator? > It's only under critical requirements that postgres necessitates a > baby sitter. And those projects don't suffer from lack of competent > administrators. And you (or your users) handle database backups and Postgresql upgrades how? From dan at tombstonezero.net Thu Feb 19 23:54:49 2015 From: dan at tombstonezero.net (Dan Sommers) Date: Fri, 20 Feb 2015 04:54:49 +0000 (UTC) Subject: What behavior would you expect? References: Message-ID: On Fri, 20 Feb 2015 07:11:13 +1100, Chris Angelico wrote: > On Fri, Feb 20, 2015 at 7:03 AM, Denis McMahon wrote: >> On the one hand, the return type of a function (when it returns, >> rather than raising an exception) should be consistent to itself, >> even if using a language where types are not declared. > > Yes, so I'd advise against having a function sometimes return a string > and sometimes a dict. Sure. But None gets a bit of a special pass > here; in a lot of languages, what you'd have is a "nullable" type (eg > in C you might declare the return value as "pointer to char", and > either return a pointer to a string, or NULL), but in Python, you > return None for the special case. So in a sense, "str or None" is not > so inconsistent. I'm an old C and assembly language programmer, but the older I get, the less I use null (in C or in Java), None (in Python), or equivalent in other languages. If a function can't perform its side effect (like renaming a file), then raise an exception. If a function called for its value can't return that value, then raise an exception; if there are no values to return, then return an empty collection. How much code do we write and read that checks for null (or None, or whatever), and how much time do we spend writing it and reading it? Readability counts. Simple is better than complex. Explicit is better than implicit. Errors should never pass silently. In the face of ambiguity, refuse the temptation to guess. (Wow! I knew I could turn to the Zen for support, but I didn't realize just how much support I would get!) See also . Dan From jsf80238 at gmail.com Fri Feb 20 00:13:07 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 19 Feb 2015 22:13:07 -0700 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: I have need to search a directory and return the name of the most recent file matching a given pattern. Given a directory with these files and timestamps, > > q.pattern1.abc Feb 13 > r.pattern1.cdf Feb 12 > s.pattern1.efg Feb 10 > t.pattern2.abc Feb 13 > u.pattern2.xyz Feb 14 > v.pattern2.efg Feb 10 > > calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc > and calling my_function("/path/to/dir", "pattern2") will return > u.pattern2.xyz. > > My question is, what would be a reasonable behavior/result/return value if: > 1. "/path/to/dir" does not exist or is not readable > 2. no files match the given pattern > > Also, what would be a reasonable name for such a function? > Thank you everyone. Taking bits of advice from each of you I tentatively have: import glob import os def order_matching_files(a_path, a_glob="*"): """Search a path for files whose names match a_glob and return a list of the full path to such files, in descending order of modification time. Ignore directories.""" previous_dir = os.getcwd() os.chdir(a_path) return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if os.path.isfile(x)] os.chdir(previous_dir) return reversed(sorted(return_list, key=os.path.getmtime)) It's a shame that glob.glob does not take an arbitrary directory as an optional argument if one does not want to scan the current directory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 20 00:16:50 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 16:16:50 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 3:54 PM, Dan Sommers wrote: > if there are no > values to return, then return an empty collection. That one makes sense only if you were going to return a collection anyway, though. If you were going to return a string, returning an empty list on failure makes no sense. Hence the notion of returning a "non-string". ChrisA From rosuav at gmail.com Fri Feb 20 00:20:32 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 16:20:32 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 4:13 PM, Jason Friedman wrote: > os.chdir(a_path) > return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if > os.path.isfile(x)] > os.chdir(previous_dir) > return reversed(sorted(return_list, key=os.path.getmtime)) > > It's a shame that glob.glob does not take an arbitrary directory as an > optional argument if one does not want to scan the current directory. If you look at the glob module's docs, you'll see how it's implemented: https://docs.python.org/2/library/glob.html You may want to use fnmatch, or possibly just prepend a path name onto your glob. That way, you wouldn't need to change directory, which is almost always a bad idea for a library function. Instead of the reversed() call, you can simply add reverse=True to the sorted() call. And since you're not making use of the original list, you could use its sort() method (which has key and reverse parameters too) to sort a bit more efficiently. But I think your code is pretty good! ChrisA From jsf80238 at gmail.com Fri Feb 20 00:21:49 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 19 Feb 2015 22:21:49 -0700 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: > def order_matching_files(a_path, a_glob="*"): > """Search a path for files whose names match a_glob > and return a list of the full path to such files, in descending > order of modification time. Ignore directories.""" > previous_dir = os.getcwd() > os.chdir(a_path) > return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if > os.path.isfile(x)] > os.chdir(previous_dir) > return reversed(sorted(return_list, key=os.path.getmtime)) > Oops, guess I'm not really returning a list, I'm returning an iterator. I should change either the comment or the return value. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan at tombstonezero.net Fri Feb 20 00:40:52 2015 From: dan at tombstonezero.net (Dan Sommers) Date: Fri, 20 Feb 2015 05:40:52 +0000 (UTC) Subject: What behavior would you expect? References: Message-ID: On Fri, 20 Feb 2015 16:16:50 +1100, Chris Angelico wrote: > On Fri, Feb 20, 2015 at 3:54 PM, Dan Sommers wrote: >> if there are no >> values to return, then return an empty collection. > > That one makes sense only if you were going to return a collection > anyway, though. If you were going to return a string, returning an > empty list on failure makes no sense. Hence the notion of returning a > "non-string". I think I came in a little late, and saw "2. no files match the given pattern," in which case I'm sticking to my story and returning an empty list. The original problem was "to search a directory and return the name of the most recent file matching a given pattern." I'd still prefer an exception to None, and we agree on that an empty string is bad because it's not a non-string and it could be too easily mistaken for a filename. From jsf80238 at gmail.com Fri Feb 20 00:49:14 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 19 Feb 2015 22:49:14 -0700 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: I think I came in a little late, and saw "2. no files match the given > pattern," in which case I'm sticking to my story and returning an empty > list. > > The original problem was "to search a directory and return the name of > the most recent file matching a given pattern." I'd still prefer an > exception to None, and we agree on that an empty string is bad because > it's not a non-string and it could be too easily mistaken for a > filename. > > Agreed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsf80238 at gmail.com Fri Feb 20 00:51:57 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 19 Feb 2015 22:51:57 -0700 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: > > I'd still advise using my_list.sort() rather than sorted(), as you > don't need to retain the original. > > Hmm. Trying to figure out what that looks like. If I understand correctly, list.sort() returns None. What would I return to the caller? > If you're going to call listdir, you probably want to use fnmatch directly. > > Got it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsf80238 at gmail.com Fri Feb 20 00:55:44 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 19 Feb 2015 22:55:44 -0700 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: > If you're going to call listdir, you probably want to use fnmatch directly. > > fnmatch seems to be silent on non-existent directories: python -c 'import fnmatch; fnmatch.fnmatch("/no/such/path", "*")' -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Feb 20 00:57:47 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 16:57:47 +1100 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 4:51 PM, Jason Friedman wrote: >> I'd still advise using my_list.sort() rather than sorted(), as you >> don't need to retain the original. >> > > Hmm. > > Trying to figure out what that looks like. > If I understand correctly, list.sort() returns None. > What would I return to the caller? Check its docs: it sorts the list in place. So you return the list, after you sort it. It's like appending to a list: None is returned and the list itself is changed. ChrisA From dan at tombstonezero.net Fri Feb 20 00:58:09 2015 From: dan at tombstonezero.net (Dan Sommers) Date: Fri, 20 Feb 2015 05:58:09 +0000 (UTC) Subject: What behavior would you expect? References: Message-ID: On Thu, 19 Feb 2015 22:51:57 -0700, Jason Friedman wrote: >> >> I'd still advise using my_list.sort() rather than sorted(), as you >> don't need to retain the original. >> >> > Hmm. > > Trying to figure out what that looks like. > If I understand correctly, list.sort() returns None. > What would I return to the caller? my_list. In the end, it'll look something like this: def f(): my_list = some_other_function_that_returns_an_unsorted_list() my_list.sort() return my_list From no.email at nospam.invalid Fri Feb 20 01:18:10 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 19 Feb 2015 22:18:10 -0800 Subject: What behavior would you expect? References: Message-ID: <87r3tlds2l.fsf@jester.gateway.pace.com> Dan Sommers writes: > I'd still prefer an exception to None, and we agree on that an empty > string is bad because it's not a non-string and it could be too easily > mistaken for a filename. Empty string would be bad. Sometimes I like to simulate an option type, by returning the value as a 1-element list if there's a value, otherwise as an empty list. So you could say filename = get_filename(...)[0] if you want an exception in the failure case, or you could do something like fs = get_filename(...) if len(fs) == 0: ... # didn't get a filename From rosuav at gmail.com Fri Feb 20 01:21:51 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 17:21:51 +1100 Subject: What behavior would you expect? In-Reply-To: <87r3tlds2l.fsf@jester.gateway.pace.com> References: <87r3tlds2l.fsf@jester.gateway.pace.com> Message-ID: On Fri, Feb 20, 2015 at 5:18 PM, Paul Rubin wrote: > Empty string would be bad. Sometimes I like to simulate an option type, > by returning the value as a 1-element list if there's a value, otherwise > as an empty list. So you could say > > filename = get_filename(...)[0] > > if you want an exception in the failure case, or you could do something > like > > fs = get_filename(...) > if len(fs) == 0: ... # didn't get a filename Bikeshedding: That could be written as simply "if not fs". :) ChrisA From no.email at nospam.invalid Fri Feb 20 02:18:04 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 19 Feb 2015 23:18:04 -0800 Subject: What behavior would you expect? References: <87r3tlds2l.fsf@jester.gateway.pace.com> Message-ID: <87mw49dpar.fsf@jester.gateway.pace.com> Chris Angelico writes: >> if len(fs) == 0: ... # didn't get a filename > Bikeshedding: That could be written as simply "if not fs". :) Yeah, in that instance you could do that. It's an unsafe practice when None is used as the no-value marker, since the empty string is a perfectly good string and also converts to boolean false. From ismahameed at gcuf.edu.pk Fri Feb 20 03:24:15 2015 From: ismahameed at gcuf.edu.pk (ismahameed at gcuf.edu.pk) Date: Fri, 20 Feb 2015 00:24:15 -0800 (PST) Subject: urgent help Message-ID: sir these errors whats mean by it warning (from warnings module): File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 br.set_handle_gzip(True) UserWarning: gzip transfer encoding is experimental! Traceback (most recent call last): File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in getLinksComlicated() File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated for file in os.listdir(dir)[1:]: WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' From breamoreboy at yahoo.co.uk Fri Feb 20 03:40:44 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 20 Feb 2015 08:40:44 +0000 Subject: urgent help In-Reply-To: References: Message-ID: On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: Would you please give your requests a meaningful subject. > sir these errors whats mean by it > warning (from warnings module): > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 > br.set_handle_gzip(True) > UserWarning: gzip transfer encoding is experimental! See http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do > > Traceback (most recent call last): > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in > getLinksComlicated() > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > for file in os.listdir(dir)[1:]: > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > Either the code needs changing so it does a wildcard search or you need to pass in 'complicated' alone. Seeing those three magic letters P H D I'll leave you to research the former :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From wxjmfauth at gmail.com Fri Feb 20 03:58:02 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 20 Feb 2015 00:58:02 -0800 (PST) Subject: python implementation of a new integer encoding algorithm. In-Reply-To: References: Message-ID: Le vendredi 20 f?vrier 2015 03:47:18 UTC+1, Denis McMahon a ?crit?: > > Perhaps it's time to take a step back and redefine the problem a bit more > clearly, because at the moment I'm not sure you're solution will ever > solve anything that needs solving. > Ditto for the Flexible String Representation and the general unicode handling. In one word: a shame. jmf From wxjmfauth at gmail.com Fri Feb 20 04:07:24 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 20 Feb 2015 01:07:24 -0800 (PST) Subject: On Python crash Message-ID: Yesterday, I spent an hour experimenting a new way on how to make Python crash. Very interesting. The cause? Unicode! jmf From ian.g.kelly at gmail.com Fri Feb 20 04:16:39 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 20 Feb 2015 02:16:39 -0700 Subject: Logging with Custom Levels not working In-Reply-To: <674667d9-a720-4d3f-979b-3ea2c2267e8b@googlegroups.com> References: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> <674667d9-a720-4d3f-979b-3ea2c2267e8b@googlegroups.com> Message-ID: On Thu, Feb 19, 2015 at 11:16 AM, Didymus wrote: > On Wednesday, February 18, 2015 at 3:16:40 PM UTC-5, Ian wrote: >> > def perror(self, message, *args, **kws): >> > """ Performance Error Message Level """ >> > # Yes, logger takes its '*args' as 'args'. >> > self._log(PERROR_NUM, message, args, **kws) >> > >> > logging.Logger.perror = perror >> >> I think you need to call self.log, not self._log. The _log method >> appears to assume that the level check has already been performed. You >> really shouldn't be calling it directly anyway, as the leading _ is an >> indication that the method is not part of the public API. > > Yes, I had tried that and get: > > Logged from file log5.py, line 21 > Traceback (most recent call last): > File "/usr/lib64/python2.7/logging/__init__.py", line 851, in emit > msg = self.format(record) > File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format > return fmt.format(record) > File "/usr/lib64/python2.7/logging/__init__.py", line 464, in format > record.message = record.getMessage() > File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting log and _log don't have the same function signature. Specifically, log takes *args and _log just takes args (as noted in the comment on the preceding line). From puruganti.ramesh at gmail.com Fri Feb 20 04:32:37 2015 From: puruganti.ramesh at gmail.com (Puruganti Ramesh) Date: Fri, 20 Feb 2015 01:32:37 -0800 (PST) Subject: Date Difference issue in Python 2.7.8 Message-ID: Hi, I have an issue in comparing dates in python 2.7.8 I have written code as below but i am getting error import datetime as dt from datetime import datetime from datetime import datetime, timedelta, date dt_str='2014-5-11' dt_strq='2014-9-11' dt_datea = datetime.strptime(dt_str, '%Y-%m-%d').date() dt_dateb = datetime.strptime(dt_strq, '%Y-%m-%d').date() dt_diff = dt_dateb - dt_datea print dt_diff.days I am getting below excption Traceback (most recent call last): File "FunctionUpdate.py", line 204, in dt_dateb = datetime.strptime(dt_strq, '%Y-%m-%d').date() File "/usr/local/lib/python2.7/_strptime.py", line 328, in _strptime data_string[found.end():]) ValueError: unconverted data remains: Kindly solve my issue Regards, Ramesh From nomail at invalid.com Fri Feb 20 04:40:10 2015 From: nomail at invalid.com (ast) Date: Fri, 20 Feb 2015 10:40:10 +0100 Subject: Bad text appearance in IDLE In-Reply-To: References: <54dc9bee$0$3046$426a34cc@news.free.fr> Message-ID: <54e700fe$0$3299$426a74cc@news.free.fr> "David H. Lipman" a ?crit dans le message de news:CZGdnVx2faf0q0LJnZ2dnUU7-UWdnZ2d at giganews.com... > > http://support.microsoft.com/kb/3013455 > KB3013455 has been found to be causing Display Font problems for Windows Vista and Server 2003. > > A partial mitigation is to use ClearType Font Smoothing. It will greatly help but will not fix > all Display Font viewing problems. > > It is recommended to remove this HotFix and then hide it from further download ( at least until > it is fixed ). > > -- thanks a lot I removed the fix and wait for Microsoft to fix it's fix From rosuav at gmail.com Fri Feb 20 04:46:01 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Feb 2015 20:46:01 +1100 Subject: Date Difference issue in Python 2.7.8 In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 8:32 PM, Puruganti Ramesh wrote: > import datetime as dt > from datetime import datetime > from datetime import datetime, timedelta, date The first two lines here are achieving nothing - the third should be all you need. > dt_str='2014-5-11' > dt_strq='2014-9-11' > > dt_datea = datetime.strptime(dt_str, '%Y-%m-%d').date() > dt_dateb = datetime.strptime(dt_strq, '%Y-%m-%d').date() > dt_diff = dt_dateb - dt_datea > print dt_diff.days > > I am getting below excption > Traceback (most recent call last): > File "FunctionUpdate.py", line 204, in > dt_dateb = datetime.strptime(dt_strq, '%Y-%m-%d').date() > File "/usr/local/lib/python2.7/_strptime.py", line 328, in _strptime > data_string[found.end():]) > ValueError: unconverted data remains: Just tried this on 2.7.3 on Debian Wheezy, and 2.7.8 on Debian Jessie, and it gave me back a date difference of 123 days. Is this definitely what's causing the issue? If so, can you cut it down to just the bare essentials? For instance, does this also fail? from datetime import datetime datetime.strptime('2014-9-11', '%Y-%m-%d') In theory, it should; my eyeballing of the code suggests no reason why the two-line version would work when your larger one doesn't. But my suspicion is that your larger code isn't triggering the problem either; your traceback points to line 204, and I'd really like to see a self-contained program that actually definitely has the problem. ChrisA From __peter__ at web.de Fri Feb 20 06:18:56 2015 From: __peter__ at web.de (Peter Otten) Date: Fri, 20 Feb 2015 12:18:56 +0100 Subject: Date Difference issue in Python 2.7.8 References: Message-ID: Puruganti Ramesh wrote: > I have an issue in comparing dates in python 2.7.8 No, you are stuck with parsing the dates, you don't get to the point of comparing them. > I am getting below excption > Traceback (most recent call last): > File "FunctionUpdate.py", line 204, in > dt_dateb = datetime.strptime(dt_strq, '%Y-%m-%d').date() > File "/usr/local/lib/python2.7/_strptime.py", line 328, in _strptime > data_string[found.end():]) > ValueError: unconverted data remains: > Look closely at the line above. You don't see anything? Well, that's a newline, I suspect ;) If you are reading dates from a file dt_strq is probably "2014-9-11\n" instead of the "2014-9-11" you and the strptime() method expect. Try removing any whitespace with dt_strq = dt_strq.strip() before passing it to datetime.strptime() From rustompmody at gmail.com Fri Feb 20 06:37:25 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 20 Feb 2015 03:37:25 -0800 (PST) Subject: What the Pythons docs means by "container" ? In-Reply-To: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: On Wednesday, February 18, 2015 at 2:51:34 AM UTC+5:30, candide wrote: > Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains : > > -------------------------- > Some objects contain references to other objects; these are called containers. > -------------------------- > > So containers contain : what a great definition! > > To be more precise, the "container" wording suggests a data structure _storing_ items somewhere and that the number of items in the container has a memory footprint. This is exacly the "container" definition given by the ISO C++ standard (cf. http://www.open-std.org/jtc1/sc22/open/n2356/lib-containers.html) : > > -------------------------- > Containers are objects that store other objects. > -------------------------- > > So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint in the contrary to the associated list : > > ---------------------------------------------- > >>> from sys import getsizeof > >>> r = range(10**6) > >>> L = list(r) > >>> getsizeof(r) > 24 > >>> getsizeof(L) > 4500056 > >>> > ---------------------------------------------- > > Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container ! > > ---------------------------------------------- > >>> from collections import Container > >>> r = range(10**6) > >>> isinstance(r, Container) > True > >>> > ---------------------------------------------- > > So, what documentation means by the generic term of "container"? > > I agree with the existence of built-in containers (list, dict and the like) and the existence of special containers provided by the collections module (most of them inheriting from a built-in container) but I can't find a precise definition explaining what is a "container object" or a "container type". > > The docs at : > > https://docs.python.org/3.2/reference/datamodel.html#object.__contains__ > > explains that a container is an object compatible with the membership test (in and not in operators). So a file is a container ? recall a file supports the in operator : > > ---------------------------------------------- > $ touch my_test.txt > $ python3.2 > Python 3.2.3 (default, Feb 28 2014, 00:22:33) > [GCC 4.7.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> 42 in open("my_test.txt") > False > >>> from collections import Container > >>> isinstance(open("my_test.txt"), Container) > False > >>> > ---------------------------------------------- +1 See https://www.python.org/dev/peps/pep-0246/ There Guido says "something much better is about to happen" Best as I know its not happened yet... [Personal note: I am currently trying to teach python and the lack of a standardized place for all protocols (in python's duck typing sense) is quite a slow-down-er for me and my class ] From greg.ewing at canterbury.ac.nz Fri Feb 20 06:56:29 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 21 Feb 2015 00:56:29 +1300 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: Jason Friedman wrote: > It's a shame that glob.glob does not take an arbitrary directory as an > optional argument if one does not want to scan the current directory. It doesn't have to -- you can give it an absolute path: >>> from glob import glob >>> glob("/usr/include/std*.h") ['/usr/include/stdarg.h', '/usr/include/stdbool.h', '/usr/include/stddef.h', '/usr/include/stdint.h', '/usr/include/stdio.h', '/usr/include/stdlib.h'] So just prepend the dir to the pattern and then pass it to glob. -- Greg From davea at davea.name Fri Feb 20 08:07:29 2015 From: davea at davea.name (Dave Angel) Date: Fri, 20 Feb 2015 08:07:29 -0500 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: <54E73191.40005@davea.name> On 02/20/2015 12:51 AM, Jason Friedman wrote: >> >> I'd still advise using my_list.sort() rather than sorted(), as you >> don't need to retain the original. >> >> > Hmm. > > Trying to figure out what that looks like. > If I understand correctly, list.sort() returns None. > What would I return to the caller? > return_list = .... return_list.sort(key=..., reverse = True) return return_list > > Oops, guess I'm not really returning a list, I'm returning an iterator. Now you'll be returning a list, I believe. -- DaveA From __peter__ at web.de Fri Feb 20 08:10:30 2015 From: __peter__ at web.de (Peter Otten) Date: Fri, 20 Feb 2015 14:10:30 +0100 Subject: Python - parsing nested information and provide it in proper format from log file References: <0097dab0-301c-42e1-a6be-b21eb5356567@googlegroups.com> Message-ID: Jay T wrote: > have some log file which has nested data which i want to filter and > provide specific for student with total counts > > Here is my log file sample: > Student name is ABC > Student age is 12 > student was late > student was late > student was late > Student name is DEF > student age is 13 > student was late > student was late > > i want to parse and show data as Student name, student age , number of > counts how many times student was late e:g Name Age TotalCount > ABC 12 3 > DEF 13 2 > > Please help me with solution that will be really grateful. What have you tried? Please show us some code. The basic idea would be to iterate over the lines and split the current line into words. If the second word is "name" and it's not the first iteration print the student's name, age, and was_late count. Then set the name variable to the new name and reset age and was_late to 0. To detect the first iteration you can set name = None before you enter the loop and then check for that value before printing: if name is not None: ... # print student data If the second word is "age" convert the 4th word to integer and set the age variable. If the second word is "was" increment the was_late counter. Remember that when the loop ends and the file was not empty you have one more student's data to print. From lynto28 at gmail.com Fri Feb 20 08:29:44 2015 From: lynto28 at gmail.com (Didymus) Date: Fri, 20 Feb 2015 05:29:44 -0800 (PST) Subject: Logging with Custom Levels not working In-Reply-To: References: <24fc6e1c-1db3-41e0-84df-18b0ca5a8c69@googlegroups.com> <674667d9-a720-4d3f-979b-3ea2c2267e8b@googlegroups.com> Message-ID: On Friday, February 20, 2015 at 4:25:50 AM UTC-5, Ian wrote: > > On Wednesday, February 18, 2015 at 3:16:40 PM UTC-5, Ian wrote: > >> > def perror(self, message, *args, **kws): > >> > """ Performance Error Message Level """ > >> > # Yes, logger takes its '*args' as 'args'. > >> > self._log(PERROR_NUM, message, args, **kws) > >> > > >> > logging.Logger.perror = perror > >> > >> I think you need to call self.log, not self._log. The _log method > >> appears to assume that the level check has already been performed. You > >> really shouldn't be calling it directly anyway, as the leading _ is an > >> indication that the method is not part of the public API. > > > > Yes, I had tried that and get: > > > > Logged from file log5.py, line 21 > > Traceback (most recent call last): > > File "/usr/lib64/python2.7/logging/__init__.py", line 851, in emit > > msg = self.format(record) > > File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format > > return fmt.format(record) > > File "/usr/lib64/python2.7/logging/__init__.py", line 464, in format > > record.message = record.getMessage() > > File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage > > msg = msg % self.args > > TypeError: not all arguments converted during string formatting > > log and _log don't have the same function signature. Specifically, log > takes *args and _log just takes args (as noted in the comment on the > preceding line). Hi, I found the problem. I wasn't setting the level on the Handlers. Need to have: fileHandler.setLevel(level) consoleHandler.setLevel(level) Once that was added to the SetLogging() function, it was printing out the correct levels. The rootLogger.setLevel(level) wasn't getting propagated to the Handlers... Thanks for all the help! Tom From jt11380 at gmail.com Fri Feb 20 08:31:44 2015 From: jt11380 at gmail.com (jt11380 at gmail.com) Date: Fri, 20 Feb 2015 05:31:44 -0800 (PST) Subject: Python - parsing nested information and provide it in proper format from log file In-Reply-To: References: <0097dab0-301c-42e1-a6be-b21eb5356567@googlegroups.com> Message-ID: <03cf1f25-b31b-4c48-96c9-be86a2ecdbc8@googlegroups.com> On Friday, February 20, 2015 at 8:11:59 AM UTC-5, Peter Otten wrote: > Jay T wrote: > > > have some log file which has nested data which i want to filter and > > provide specific for student with total counts > > > > Here is my log file sample: > > Student name is ABC > > Student age is 12 > > student was late > > student was late > > student was late > > Student name is DEF > > student age is 13 > > student was late > > student was late > > > > i want to parse and show data as Student name, student age , number of > > counts how many times student was late e:g Name Age TotalCount > > ABC 12 3 > > DEF 13 2 > > > > Please help me with solution that will be really grateful. > > What have you tried? Please show us some code. > > The basic idea would be to iterate over the lines and split the current line > into words. > > If the second word is "name" and it's not the first iteration print the > student's name, age, and was_late count. Then set the name variable to the > new name and reset age and was_late to 0. To detect the first iteration you > can set > > name = None > > before you enter the loop and then check for that value before printing: > > if name is not None: > ... # print student data > > If the second word is "age" convert the 4th word to integer and set the age > variable. > > If the second word is "was" increment the was_late counter. > > Remember that when the loop ends and the file was not empty you have one > more student's data to print. I tried to implent below code and got stucked how to do nested loop to count instead doing another logic and parsing: import re def GetName(input_string): myName=input_string.split() myName1= myName[1] return myName1 def GetAge(input_string): myAge=input_string.split() myAge1= myAge[2] return myAge1 file = open('mylogfile') log_data = file.readlines() print 'entered' for eachline in log_data: input_string = eachline if 'name' in input_string: sometextval = GetName(input_string) print "name", sometextval if 'Age' in input_string: sometextval2 = GetAge(input_string) print "Age", sometextval2 Now get stuck to get count for total_late time as it is part of name, age so how to write logic which counts as a part of group. any help will be grateful. -J From __peter__ at web.de Fri Feb 20 09:27:59 2015 From: __peter__ at web.de (Peter Otten) Date: Fri, 20 Feb 2015 15:27:59 +0100 Subject: Python - parsing nested information and provide it in proper format from log file References: <0097dab0-301c-42e1-a6be-b21eb5356567@googlegroups.com> <03cf1f25-b31b-4c48-96c9-be86a2ecdbc8@googlegroups.com> Message-ID: jt11380 at gmail.com wrote: > On Friday, February 20, 2015 at 8:11:59 AM UTC-5, Peter Otten wrote: >> Jay T wrote: >> >> > have some log file which has nested data which i want to filter and >> > provide specific for student with total counts >> > >> > Here is my log file sample: >> > Student name is ABC >> > Student age is 12 >> > student was late >> > student was late >> > student was late >> > Student name is DEF >> > student age is 13 >> > student was late >> > student was late >> > >> > i want to parse and show data as Student name, student age , number of >> > counts how many times student was late e:g Name Age TotalCount >> > ABC 12 3 >> > DEF 13 2 >> > >> > Please help me with solution that will be really grateful. >> >> What have you tried? Please show us some code. >> >> The basic idea would be to iterate over the lines and split the current >> line into words. >> >> If the second word is "name" and it's not the first iteration print the >> student's name, age, and was_late count. Then set the name variable to >> the new name and reset age and was_late to 0. To detect the first >> iteration you can set >> >> name = None >> >> before you enter the loop and then check for that value before printing: >> >> if name is not None: >> ... # print student data >> >> If the second word is "age" convert the 4th word to integer and set the >> age variable. >> >> If the second word is "was" increment the was_late counter. >> >> Remember that when the loop ends and the file was not empty you have one >> more student's data to print. > Now get stuck to get count for total_late time as it is part of name, age > so how to write logic which counts as a part of group. > > any help will be grateful. Try to write code that does what I describe in my outline. Initialise name before the loop and dump the collected data when you encounter a new name. name = None total_late = 0 age = "unknown" with open("student.txt") as instream: for line in instream: words = line.split() if words[1] == "name": if name is not None: print name, age, total_late name = " ".join(words[3:]) age = "unknown" total_late = 0 elif words[1] == "was": total_late += 1 elif words[1] == "age": age = int(words[3]) else: print "don't know what to do with line %r" %line if name is not None: print name, age, total_late Checking whole words has the advantage that there will be no match if the string "name" or "age" is part of the student's name. > I tried to implent below code and got stucked how to do nested loop to > count instead doing another logic and parsing: > > import re > def GetName(input_string): > myName=input_string.split() > myName1= myName[1] That's the wrong index. > return myName1 > def GetAge(input_string): > myAge=input_string.split() > myAge1= myAge[2] That's the wrong index. > return myAge1 In general you should test your functions independently from the whole program. That way you can build on known-good components and thus reduce the area where to look for remaining bugs. > file = open('mylogfile') > log_data = file.readlines() The file is probably short enough that it doesn't matter here, but iterating over the file directly is good habit to get into. Example: with open("mylogfile") as log_data: for eachline in log_data: ... > print 'entered' > for eachline in log_data: > input_string = eachline > if 'name' in input_string: > sometextval = GetName(input_string) > print "name", sometextval > if 'Age' in input_string: This test is problematic because Python takes case into account when comparing strings: >>> "age" == "AGE" False >>> s = "RAGE" >>> "age" in s False If case isn't consistent you should convert the string to lowercase: >>> "age" == "AGE".lower() True >>> "age" in s.lower() True > sometextval2 = GetAge(input_string) > print "Age", sometextval2 > From jldunn2000 at gmail.com Fri Feb 20 09:43:21 2015 From: jldunn2000 at gmail.com (loial) Date: Fri, 20 Feb 2015 06:43:21 -0800 (PST) Subject: Python path on windows Message-ID: On Linux we use #!/usr/bin/env python At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. What is the equivalent functionality in Windows? From davea at davea.name Fri Feb 20 09:51:53 2015 From: davea at davea.name (Dave Angel) Date: Fri, 20 Feb 2015 09:51:53 -0500 Subject: Python path on windows In-Reply-To: References: Message-ID: <54E74A09.1060906@davea.name> On 02/20/2015 09:43 AM, loial wrote: > On Linux we use > #!/usr/bin/env python > > At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. > > What is the equivalent functionality in Windows? > Depends on Python version. In newer versions, there is a py.exe file which they put on their path. Then in the registry is a two-stage association between the .py and .pyw extensions and the above executable. Finally, when py.exe starts, it reads that first (shebang) line, and decides which python interpreter to actually use. One problem I've heard about is that if someone installs 2.7 on top of 3.x, it can wipe out the file associations, and mess this stuff up. The py.exe can be separately installed, to use on older python versions. This all happened after I cut loose from Windows. -- DaveA From ian.g.kelly at gmail.com Fri Feb 20 09:54:11 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 20 Feb 2015 07:54:11 -0700 Subject: Python path on windows In-Reply-To: References: Message-ID: On Feb 20, 2015 7:46 AM, "loial" wrote: > > On Linux we use > #!/usr/bin/env python > > At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. > > What is the equivalent functionality in Windows? https://docs.python.org/3/using/windows.html#launcher Note that while the launcher can be used with any version of Python, it is only packaged with 3.3+. -------------- next part -------------- An HTML attachment was scrubbed... URL: From al.basili at gmail.com Fri Feb 20 09:59:08 2015 From: al.basili at gmail.com (alb) Date: 20 Feb 2015 14:59:08 GMT Subject: pypandoc and restructured text Message-ID: Hi everyone, I'm trying to convert restructured text to latex with pandoc and it seems to me there's something not correctly working. I have the following text: .. figure:: picture.png :scale: 50 % :alt: map to buried treasure This is the caption of the figure (a simple paragraph). The legend consists of all elements after the caption. In this case, the legend consists of this paragraph and the following table: +-----------------------+-----------------------+ | Symbol | Meaning | +=======================+=======================+ | .. image:: tent.png | Campground | +-----------------------+-----------------------+ | .. image:: waves.png | Lake | +-----------------------+-----------------------+ | .. image:: peak.png | Mountain | +-----------------------+-----------------------+ which is essentially copied from here: http://docutils.sourceforge.net/docs/ref/rst/directives.html#figure But unfortunately the pypandoc silently ignores everything. The same happens with pandoc directly. My installation is the following: pandoc 1.5.1.1 python 2.6.6 debian squeeze Any idea why? Should I upgrade somehow beyond what the debian repository delivers? Al p.s.: I understand that this is potentially not related to python and more related to the package, but there's a chance somebody here already had the same problem. -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From jldunn2000 at gmail.com Fri Feb 20 10:16:11 2015 From: jldunn2000 at gmail.com (loial) Date: Fri, 20 Feb 2015 07:16:11 -0800 (PST) Subject: Python path on windows In-Reply-To: References: Message-ID: On Friday, February 20, 2015 at 2:54:26 PM UTC, Ian wrote: > On Feb 20, 2015 7:46 AM, "loial" wrote: > > > > > > On Linux we use > > > #!/usr/bin/env python > > > > > > At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. > > > > > > What is the equivalent functionality in Windows? > > https://docs.python.org/3/using/windows.html#launcher > > Note that while the launcher can be used with any version of Python, it is only packaged with 3.3+. Vesrion is 2.6 From wxjmfauth at gmail.com Fri Feb 20 10:20:30 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 20 Feb 2015 07:20:30 -0800 (PST) Subject: Python path on windows In-Reply-To: References: Message-ID: <0ffccc89-970d-4504-a6d7-d3c4d5788c58@googlegroups.com> Le vendredi 20 f?vrier 2015 15:43:38 UTC+1, loial a ?crit?: > On Linux we use > #!/usr/bin/env python > Which one of theese? >>> 'abc?def'.upper() 'ABC?DEF' >>> 'abc?def'.upper() 'ABCSSDEF' >>> 'abc?def'.upper() 'ABC\xe1DEF' >>> From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 20 11:07:30 2015 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 20 Feb 2015 17:07:30 +0100 Subject: pypandoc and restructured text In-Reply-To: References: Message-ID: On 20.02.2015 15:59, alb wrote: > > My installation is the following: > pandoc 1.5.1.1 > python 2.6.6 > debian squeeze > > Any idea why? Should I upgrade somehow beyond what the debian repository > delivers? > I have pandoc 1.12.2.1 and it recognizes the figure directive just fine (tested with html output so I cannot say anything about LaTeX). I've found an old forum post from 2011: https://groups.google.com/forum/#!topic/pandoc-discuss/OmGYDycaMjs confirming that the figure directive was not supported at that time. So, yes, I think upgrading pandoc could help. Best, Wolfgang From zrdunlap at gmail.com Fri Feb 20 12:16:11 2015 From: zrdunlap at gmail.com (Zach Dunlap) Date: Fri, 20 Feb 2015 09:16:11 -0800 (PST) Subject: Python Requests logging 401 immediately before 200 Message-ID: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> Hi everyone, I'm using MarkLogic and have a Python script set up to check for documents using Requests (http://docs.python-requests.org/en/latest/) and I have logging included in the script. I have logging set to the DEBUG level. When I set the script to simple search and return the HTTP status code for items it works, but when I look at what is generated by logging I see the following for everything: INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211 DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327 As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time. When I search for a document using something like CURL for instance, I only get the 200 status code which seems correct to me. Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about? Thank you for your help! Here's a snipped of my code in case it helps: import requests import logging # initiate logging log_path = os.path.join('my_log.log') logging.basicConfig(filename=log_path,level=logging.DEBUG) # make request r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD')) print r.status_code From esj at harvee.org Fri Feb 20 12:22:08 2015 From: esj at harvee.org (Eric S. Johansson) Date: Fri, 20 Feb 2015 12:22:08 -0500 Subject: Accessible tools In-Reply-To: <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> <54E5B61A.903@wielicki.name> <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> Message-ID: <54E76D40.9030901@harvee.org> On 2/19/2015 10:33 AM, Bryan Duarte wrote: > Thank you jwi, and Jacob, > > I took a look at that posting and it seems pretty unique. I am not much interested in the speech driven development, but I am very interested in developing an accessible IDE. Well you should be because it looks like an aural interface (uses speech instead of keyboards) uses the same kinds of data to present to either a text to speech or speech recognition driven environment. > A professor and I have been throwing around the idea of developing a completely text based IDE. There are a lot of reasons this could be beneficial to a blind developer and maybe even some sighted developers who are comfortable in the terminal. The idea would be really just to provide a way of easily navigating blocks of code using some kind of tabular formatting, and being able to collapse blocks of code and hearing from a high level information about the code within. All tools and features would obviously be spoken or output in some kind of audio manor. I've been working with another professor working on some of these issues as well. His focus has been mostly blind young adults in India. come up with some pretty cool concepts that looks very usable. The challenge now is to make them work and, quite frankly monetize the effort to pay for the development. Again, this shows the similarities in functionality used by both speech recognition and text-to-speech. All I care about is text and what I can say. We're now working with constructs such as with-open, argument by number, plaintext symbol names (with bidirectional transform to and from code form), guided construct generation for things like classes, methods, comprehensions etc. All of these things would be useful to handed programmers as well as a way of accelerating co-creation and editing. Unfortunately, like with disabled people stove piping text-to-speech versus speech recognition, handed developers stovepipe keyboard interfaces and don't really think about what they are trying to do, only how they are doing it. Yes yes, it's a broadbrush that you can probably slap me with. :-) > > Oh and before I forget does anyone know how to contact Eric who was developing that accessible speech driven IDE? Thanks Well, you could try looking in a mirror and speaking my name three times at midnight But you would get better results if you used my non-mailing list email address. esj at eggo.org. --- eric From rosuav at gmail.com Fri Feb 20 12:30:21 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Feb 2015 04:30:21 +1100 Subject: Python Requests logging 401 immediately before 200 In-Reply-To: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> References: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> Message-ID: On Sat, Feb 21, 2015 at 4:16 AM, Zach Dunlap wrote: > INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211 > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327 > > As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time. > > Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about? > > r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD')) > Short explanation: It's part of the protocol. You're using Digest Auth, so what's happening here is that a request is sent without the authentication, and the inevitable 401 response is the signal that the request should be re-sent with credentials. There's a massive debate as to whether or not it's correct to send unsolicited credentials. On the one hand, this "401 then 200" pattern takes extra internet round-trips (not that that's a big deal with localhost, but it's the same issue everywhere) and puts extra load on both server and client; but on the other hand, nobody wants their credentials sent to the wrong server, and it's not always easy to tell when they'll be needed. Without the 401, you can't know whether or not you need to authenticate, so if once you start sending unsolicited credentials, you basically have to keep on doing so. If you don't have a problem with performance or latency, then consider this to be nothing more than a bit of log file spam. I'm not sure if the Python requests module can be easily configured to send credentials on the first query, but my advice is: don't even bother looking into that unless you have need to. All the best! ChrisA From zrdunlap at gmail.com Fri Feb 20 12:39:54 2015 From: zrdunlap at gmail.com (Zach Dunlap) Date: Fri, 20 Feb 2015 09:39:54 -0800 (PST) Subject: Python Requests logging 401 immediately before 200 In-Reply-To: References: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> Message-ID: <76cab98b-fc5a-4455-9516-f5dfa4766c4b@googlegroups.com> On Friday, February 20, 2015 at 12:31:00 PM UTC-5, Chris Angelico wrote: > On Sat, Feb 21, 2015 at 4:16 AM, Zach Dunlap wrote: > > INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost > > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211 > > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327 > > > > As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time. > > > > Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about? > > > > r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD')) > > > > Short explanation: It's part of the protocol. > > You're using Digest Auth, so what's happening here is that a request > is sent without the authentication, and the inevitable 401 response is > the signal that the request should be re-sent with credentials. > > There's a massive debate as to whether or not it's correct to send > unsolicited credentials. On the one hand, this "401 then 200" pattern > takes extra internet round-trips (not that that's a big deal with > localhost, but it's the same issue everywhere) and puts extra load on > both server and client; but on the other hand, nobody wants their > credentials sent to the wrong server, and it's not always easy to tell > when they'll be needed. Without the 401, you can't know whether or not > you need to authenticate, so if once you start sending unsolicited > credentials, you basically have to keep on doing so. > > If you don't have a problem with performance or latency, then consider > this to be nothing more than a bit of log file spam. I'm not sure if > the Python requests module can be easily configured to send > credentials on the first query, but my advice is: don't even bother > looking into that unless you have need to. > > All the best! > > ChrisA Thanks very much for this it's extremely helpful! This clears it up and I think I will ignore it as log spam as you suggest. From ian.g.kelly at gmail.com Fri Feb 20 13:25:05 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 20 Feb 2015 11:25:05 -0700 Subject: Python path on windows In-Reply-To: References: Message-ID: On Fri, Feb 20, 2015 at 8:16 AM, loial wrote: > On Friday, February 20, 2015 at 2:54:26 PM UTC, Ian wrote: >> On Feb 20, 2015 7:46 AM, "loial" wrote: >> >> > >> >> > On Linux we use >> >> > #!/usr/bin/env python >> >> > >> >> > At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. >> >> > >> >> > What is the equivalent functionality in Windows? >> >> https://docs.python.org/3/using/windows.html#launcher >> >> Note that while the launcher can be used with any version of Python, it is only packaged with 3.3+. > > Vesrion is 2.6 You can install multiple versions of Python on the same system without conflict. If you install Python 3.4 also, then you'll have the launcher. I don't know if there's an easy way to install just the launcher. From breamoreboy at yahoo.co.uk Fri Feb 20 13:31:20 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 20 Feb 2015 18:31:20 +0000 Subject: Python path on windows In-Reply-To: References: Message-ID: On 20/02/2015 15:16, loial wrote: > On Friday, February 20, 2015 at 2:54:26 PM UTC, Ian wrote: >> On Feb 20, 2015 7:46 AM, "loial" wrote: >> >>> >> >>> On Linux we use >> >>> #!/usr/bin/env python >> >>> >> >>> At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. >> >>> >> >>> What is the equivalent functionality in Windows? >> >> https://docs.python.org/3/using/windows.html#launcher >> >> Note that while the launcher can be used with any version of Python, it is only packaged with 3.3+. > > Vesrion is 2.6 > See https://bitbucket.org/vinay.sajip/pylauncher As an aside it looks from the formatting above as if you're using google groups to post. If that is the case would you please access this list via https://mail.python.org/mailman/listinfo/python-list or read and action this https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing double line spacing and single line paragraphs, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Fri Feb 20 13:34:56 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 20 Feb 2015 18:34:56 +0000 Subject: Python Requests logging 401 immediately before 200 In-Reply-To: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> References: <6349a85a-f13e-47b1-a5c1-6c87cd96fa2e@googlegroups.com> Message-ID: On 20/02/2015 17:16, Zach Dunlap wrote: > Hi everyone, > > I'm using MarkLogic A name that frightens me to death, what idiot thought of that? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ian.g.kelly at gmail.com Fri Feb 20 13:38:58 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 20 Feb 2015 11:38:58 -0700 Subject: What the Pythons docs means by "container" ? In-Reply-To: References: <71e8463f-2a60-4fb0-a5b7-0ca7cd3efece@googlegroups.com> Message-ID: On Fri, Feb 20, 2015 at 4:37 AM, Rustom Mody wrote: > See https://www.python.org/dev/peps/pep-0246/ > > There Guido says "something much better is about to happen" > > Best as I know its not happened yet... Well, since that PEP was rejected we now have ABCs. Although PEP 3119 wasn't written until a couple of years later, so who knows if that's what he had in mind. From jacob at blindza.co.za Fri Feb 20 13:39:15 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Fri, 20 Feb 2015 20:39:15 +0200 Subject: Accessible tools Message-ID: <479CFB67BB2640C09ADB16742A547172@JakesPC> Eric, issue is that with screenreaders, we're generally way more into navigating code and interface character by character/by keyboard, so , yes, keeping interface relatively simple is a good thing, but, we also would prefer to primarily keep all interface elements to make use of standard UI controls, and make sure tab index/order is suitable/relevant at times, etc. etc. As in, I think we'd primarily want to avoid having to use a mouse at all if possible, but anyway. Stay well Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." ----- Original Message ----- From: "Eric S. Johansson" To: Sent: Friday, February 20, 2015 7:22 PM Subject: Re: Accessible tools > > On 2/19/2015 10:33 AM, Bryan Duarte wrote: >> Thank you jwi, and Jacob, >> >> I took a look at that posting and it seems pretty unique. I am not much >> interested in the speech driven development, but I am very interested in >> developing an accessible IDE. > > Well you should be because it looks like an aural interface (uses speech > instead of keyboards) uses the same kinds of data to present to either a > text to speech or speech recognition driven environment. >> A professor and I have been throwing around the idea of developing a >> completely text based IDE. There are a lot of reasons this could be >> beneficial to a blind developer and maybe even some sighted developers >> who are comfortable in the terminal. The idea would be really just to >> provide a way of easily navigating blocks of code using some kind of >> tabular formatting, and being able to collapse blocks of code and hearing >> from a high level information about the code within. All tools and >> features would obviously be spoken or output in some kind of audio manor. > I've been working with another professor working on some of these issues > as well. His focus has been mostly blind young adults in India. come up > with some pretty cool concepts that looks very usable. The challenge now > is to make them work and, quite frankly monetize the effort to pay for the > development. > > Again, this shows the similarities in functionality used by both speech > recognition and text-to-speech. All I care about is text and what I can > say. We're now working with constructs such as with-open, argument by > number, plaintext symbol names (with bidirectional transform to and from > code form), guided construct generation for things like classes, methods, > comprehensions etc. > > All of these things would be useful to handed programmers as well as a way > of accelerating co-creation and editing. Unfortunately, like with disabled > people stove piping text-to-speech versus speech recognition, handed > developers stovepipe keyboard interfaces and don't really think about what > they are trying to do, only how they are doing it. > > Yes yes, it's a broadbrush that you can probably slap me with. :-) >> >> Oh and before I forget does anyone know how to contact Eric who was >> developing that accessible speech driven IDE? Thanks > > Well, you could try looking in a mirror and speaking my name three times > at midnight But you would get better results if you used my non-mailing > list email address. esj at eggo.org. > > --- eric > -- > https://mail.python.org/mailman/listinfo/python-list > From johnnybureddu at gmx.com Fri Feb 20 15:20:54 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Fri, 20 Feb 2015 12:20:54 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <0f3994c2-b10b-4251-b8ff-7555128b6a52@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Fri Feb 20 15:24:07 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Fri, 20 Feb 2015 12:24:07 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: <25c6856a-4f51-4f8e-86a9-8fff2e9a8e76@googlegroups.com> -- redacted -- From johnnybureddu at gmx.com Fri Feb 20 15:24:39 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Fri, 20 Feb 2015 12:24:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: -- redacted -- From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 20 15:32:16 2015 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 20 Feb 2015 21:32:16 +0100 Subject: Python path on windows In-Reply-To: References: Message-ID: On 20.02.2015 19:25, Ian Kelly wrote: > On Fri, Feb 20, 2015 at 8:16 AM, loial wrote: >> On Friday, February 20, 2015 at 2:54:26 PM UTC, Ian wrote: >>> On Feb 20, 2015 7:46 AM, "loial" wrote: >>> >>>> >>> >>>> On Linux we use >>> >>>> #!/usr/bin/env python >>> >>>> >>> >>>> At the start of scripts to ensure that the python executable used is the one defined in the PATH variable, rather than hardcoding a path to the python executable. >>> >>>> >>> >>>> What is the equivalent functionality in Windows? >>> >>> https://docs.python.org/3/using/windows.html#launcher >>> >>> Note that while the launcher can be used with any version of Python, it is only packaged with 3.3+. >> >> Vesrion is 2.6 > > You can install multiple versions of Python on the same system without > conflict. If you install Python 3.4 also, then you'll have the > launcher. I don't know if there's an easy way to install just the > launcher. https://bitbucket.org/vinay.sajip/pylauncher/overview has the standalone launcher (msi installers are in the downloads section). From no.email at nospam.invalid Fri Feb 20 16:17:54 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 20 Feb 2015 13:17:54 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> Message-ID: <87a908e0zh.fsf@jester.gateway.pace.com> Ben Finney writes: > I don't know of a free-software concurrent RDBMS which can be considered > lighter than that. (No, MySQL doesn't count; its concurrency is > *unreliable* and it commonly loses data silently. Don't use MySQL.) I thought they fixed MySQL transactions years ago, with the InnoDB engine. For some reason it's not the default, so you have to turn it on explicitly: is there more to it than that? For stuff like browser bookmarks or other typical embedded database purposes, I don't see why SQL or relations are needed. Berkeley DB is a transactional key-value store that's been around for decades and is way simpler than SQLite, and there's other things like that too. SQLite always seemed bloated (from the embedded NoSQL point of view) and fragile to me, and the vendor plays an annoying anti-forking trick, which is that the code is released but the developers' test suite is secret and proprietary (can be licensed from them for big bucks). So if you want to make your own version of SQLite you have to either pay for the test suite, or have much less reliability assurance for your patched version than the vendor has for their version. Add that Sqlite is written in C (think of naked whirling razor blades) and you've got a pretty serious disincentive against modification. From python.list at tim.thechases.com Fri Feb 20 16:31:58 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 20 Feb 2015 15:31:58 -0600 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <87a908e0zh.fsf@jester.gateway.pace.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> Message-ID: <20150220153158.53fdc506@bigbox.christie.dr> On 2015-02-20 13:17, Paul Rubin wrote: > For stuff like browser bookmarks or other typical embedded database > purposes, I don't see why SQL or relations are needed. Berkeley DB > is a transactional key-value store that's been around for decades > and is way simpler than SQLite, and there's other things like that > too. Well, for Steven's purposes, both fail ungracefully on network shares. From my understanding, BDB fails even harder (whereas the sqlite doesn't so much fail as keep a locked write transaction locked if it fails at the wrong time, so you have to force an unlock) And all said, sqlite doesn't add that much overhead compared to the many other libraries I've used that aren't nearly so well-tested. -tkc From breamoreboy at yahoo.co.uk Fri Feb 20 16:44:28 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 20 Feb 2015 21:44:28 +0000 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <87a908e0zh.fsf@jester.gateway.pace.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> Message-ID: On 20/02/2015 21:17, Paul Rubin wrote: > Ben Finney writes: >> I don't know of a free-software concurrent RDBMS which can be considered >> lighter than that. (No, MySQL doesn't count; its concurrency is >> *unreliable* and it commonly loses data silently. Don't use MySQL.) > > I thought they fixed MySQL transactions years ago, with the InnoDB > engine. For some reason it's not the default, so you have to turn it on > explicitly: is there more to it than that? > > For stuff like browser bookmarks or other typical embedded database > purposes, I don't see why SQL or relations are needed. Berkeley DB is a > transactional key-value store that's been around for decades and is way > simpler than SQLite, and there's other things like that too. I thought I recognised the name, so a quick search and found it was deprecated in Python 2.6, removed from 3.0. Supported bindings available here https://www.jcea.es/programacion/pybsddb.htm for anybody who's interested. > > SQLite always seemed bloated (from the embedded NoSQL point of view) and > fragile to me, and the vendor plays an annoying anti-forking trick, > which is that the code is released but the developers' test suite is > secret and proprietary (can be licensed from them for big bucks). So if > you want to make your own version of SQLite you have to either pay for > the test suite, or have much less reliability assurance for your patched > version than the vendor has for their version. Add that Sqlite is > written in C (think of naked whirling razor blades) and you've got a > pretty serious disincentive against modification. > Thanks for the above. I've been meaning to take a look at how SQLite is tested for months if not years. That saves me the trouble :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ethan at stoneleaf.us Fri Feb 20 17:10:34 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 20 Feb 2015 14:10:34 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <87a908e0zh.fsf@jester.gateway.pace.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> Message-ID: <54E7B0DA.7060303@stoneleaf.us> On 02/20/2015 01:17 PM, Paul Rubin wrote: > SQLite always seemed bloated (from the embedded NoSQL point of view) and > fragile to me, and the vendor plays an annoying anti-forking trick, > which is that the code is released but the developers' test suite is > secret and proprietary (can be licensed from them for big bucks). Wow, really? I had just started playing with it, but I don't think I'll bother now. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From bcddd214 at gmail.com Fri Feb 20 18:30:40 2015 From: bcddd214 at gmail.com (Brad s) Date: Fri, 20 Feb 2015 15:30:40 -0800 (PST) Subject: subprocess command fails Message-ID: # cat makekeys.py #!/usr/bin/python3.4 import subprocess import sys import string import os.path import datetime import shlex from time import gmtime, strftime from subprocess import Popen, PIPE, STDOUT pretime = strftime("%Y%m%d%H", gmtime()) time = datetime.datetime.strptime(pretime,'%Y%m%d%H') print (time) plustime = datetime.timedelta(days=730) timeadd = (time + plustime) str(timeadd) #ndate = datetime.strptime(timeadd, '%Y%m%d%H') #timeadd = timeadd.replace(tzinfo=UTC()) print (timeadd) dname = input("Enter the domain to configure keys for? ") if os.path.exists(dname+".external.signed"): os.remove(dname+".external.signed") #os.remove(dname+".external") os.remove(dname+".ksk.key") os.remove(dname+".zsk.key") os.remove(dname+".ksk.private") os.remove(dname+".zsk.private") fd = open( dname+".external", 'w') fd.write("$TTL 86400\n") fd.write("$ORIGIN "+dname+".\n") fd.write("@ 1D IN SOA yoda.ex-mailer.com. admin@"+dname+".(\n") fd.write(" "+strftime("%Y%m%d%H", gmtime())+"\n") #fd.write(" "+repr(timeadd)+"\n") fd.write(" 3h\n") fd.write(" 1h\n") fd.write(" 1w\n") fd.write(" 1h)\n") fd.write(" IN NS yoda.ex-mailer.com.\n") fd.write(" IN NS r2d2.ex-mailer.com.\n") fd.write(dname+". IN TXT v=spf1 mx a:r2d2.ex-mailer.com -all\n") fd.write(dname+". MX 0 r2d2.ex-mailer.com.\n") fd.write("mail."+dname+". IN A 107.191.60.48\n") fd.write("$include /usr/local/etc/namedb/K"+dname+".zsk.key ; ZSK\n") fd.write("$include /usr/local/etc/namedb/K"+dname+".ksk.key ; KSK\n") fd.close() result = subprocess.check_output(["dnssec-keygen", "-f", "KSK", "-r", "/dev/urandom", "-a", "RSASHA256", "-b", "2048", "-n", "ZONE", dname]) result_utf8 = result.decode("utf-8").strip() mylist = list(result_utf8) print (mylist[0]) listlen= len(mylist) array = list() listlen -= 11 i = 0 while( i < listlen ): #if mylist != '\n' ^ mylist != '': array.insert(i, mylist[i]) i = i + 1 combined = "".join(array) print ('combined') print (combined) fmove = subprocess.call(["mv", result_utf8+".key",combined +".ksk.key"]) fmove = subprocess.call(["mv", result_utf8+".private",combined +".ksk.private"]) zresult = subprocess.check_output(["dnssec-keygen","-r","/dev/urandom","-a","RSASHA256","-b","2048","-n","ZONE", dname]) zresult_utf8 = zresult.decode("utf-8").strip() myzlist = list(zresult_utf8) print (myzlist[0]) zlistlen= len(myzlist) zarray = list() zlistlen -= 11 zi = 0 while( zi References: Message-ID: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: > On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: > > Would you please give your requests a meaningful subject. > > > sir these errors whats mean by it > > warning (from warnings module): > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 > > br.set_handle_gzip(True) > > UserWarning: gzip transfer encoding is experimental! > > See > http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do > > > > > Traceback (most recent call last): > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in > > getLinksComlicated() > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > > for file in os.listdir(dir)[1:]: > > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > > > > Either the code needs changing so it does a wildcard search or you need > to pass in 'complicated' alone. Seeing those three magic letters P H D > I'll leave you to research the former :) > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence sir what mean by the following errors: Traceback (most recent call last): File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in links = getExpertInfoLinks() File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks fid = open("health-experts.htm","rb") IOError: [Errno 2] No such file or directory: 'health-experts.htm' From ms.isma222 at gmail.com Fri Feb 20 19:16:48 2015 From: ms.isma222 at gmail.com (ms.isma222 at gmail.com) Date: Fri, 20 Feb 2015 16:16:48 -0800 (PST) Subject: urgent help In-Reply-To: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> Message-ID: <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> On Saturday, February 21, 2015 at 8:14:43 AM UTC+8, ms.is... at gmail.com wrote: > On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: > > On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: > > > > Would you please give your requests a meaningful subject. > > > > > sir these errors whats mean by it > > > warning (from warnings module): > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 > > > br.set_handle_gzip(True) > > > UserWarning: gzip transfer encoding is experimental! > > > > See > > http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do > > > > > > > > Traceback (most recent call last): > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in > > > getLinksComlicated() > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > > > for file in os.listdir(dir)[1:]: > > > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > > > > > > > Either the code needs changing so it does a wildcard search or you need > > to pass in 'complicated' alone. Seeing those three magic letters P H D > > I'll leave you to research the former :) > > > > -- > > My fellow Pythonistas, ask not what our language can do for you, ask > > what you can do for our language. > > > > Mark Lawrence > > > sir what mean by the following errors: > Traceback (most recent call last): > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in > links = getExpertInfoLinks() > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks > fid = open("health-experts.htm","rb") > IOError: [Errno 2] No such file or directory: 'health-experts.htm' File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 149, in getForums() File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 143, in getForums soup = BeautifulSoup(open(file,"r").read()) File "C:\Python27\lib\codecs.py", line 878, in open file = __builtin__.open(filename, mode, buffering) IOError: [Errno 2] No such file or directory: 'links.htm From ms.isma222 at gmail.com Fri Feb 20 19:20:46 2015 From: ms.isma222 at gmail.com (ms.isma222 at gmail.com) Date: Fri, 20 Feb 2015 16:20:46 -0800 (PST) Subject: urgent help In-Reply-To: <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> Message-ID: <73f29253-aeea-4220-b89c-59e61cbfb418@googlegroups.com> On Saturday, February 21, 2015 at 8:17:06 AM UTC+8, ms.is... at gmail.com wrote: > On Saturday, February 21, 2015 at 8:14:43 AM UTC+8, ms.is... at gmail.com wrote: > > On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: > > > On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: > > > > > > Would you please give your requests a meaningful subject. > > > > > > > sir these errors whats mean by it > > > > warning (from warnings module): > > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 > > > > br.set_handle_gzip(True) > > > > UserWarning: gzip transfer encoding is experimental! > > > > > > See > > > http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do > > > > > > > > > > > Traceback (most recent call last): > > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in > > > > getLinksComlicated() > > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > > > > for file in os.listdir(dir)[1:]: > > > > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > > > > > > > > > > Either the code needs changing so it does a wildcard search or you need > > > to pass in 'complicated' alone. Seeing those three magic letters P H D > > > I'll leave you to research the former :) > > > > > > -- > > > My fellow Pythonistas, ask not what our language can do for you, ask > > > what you can do for our language. > > > > > > Mark Lawrence > > > > > > sir what mean by the following errors: > > Traceback (most recent call last): > > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in > > links = getExpertInfoLinks() > > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks > > fid = open("health-experts.htm","rb") > > IOError: [Errno 2] No such file or directory: 'health-experts.htm' > > File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 149, in > getForums() > File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 143, in getForums > soup = BeautifulSoup(open(file,"r").read()) > File "C:\Python27\lib\codecs.py", line 878, in open > file = __builtin__.open(filename, mode, buffering) > IOError: [Errno 2] No such file or directory: 'links.htm Traceback (most recent call last): File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 106, in getLinksComlicated() File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated for file in os.listdir(dir)[1:]: WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' From davea at davea.name Fri Feb 20 19:45:16 2015 From: davea at davea.name (Dave Angel) Date: Fri, 20 Feb 2015 19:45:16 -0500 Subject: urgent help In-Reply-To: <73f29253-aeea-4220-b89c-59e61cbfb418@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> <73f29253-aeea-4220-b89c-59e61cbfb418@googlegroups.com> Message-ID: <54E7D51C.4050908@davea.name> On 02/20/2015 07:20 PM, ms.isma222 at gmail.com wrote: > On Saturday, February 21, 2015 at 8:17:06 AM UTC+8, ms.is... at gmail.com wrote: >> On Saturday, February 21, 2015 at 8:14:43 AM UTC+8, ms.is... at gmail.com wrote: >>> On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: >>>> On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: >>>> >>>> Would you please give your requests a meaningful subject. >>>> >>>>> sir these errors whats mean by it >>>>> warning (from warnings module): >>>>> File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 >>>>> br.set_handle_gzip(True) >>>>> UserWarning: gzip transfer encoding is experimental! >>>> >>>> See >>>> http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do >>>> >>>>> >>>>> Traceback (most recent call last): >>>>> File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in >>>>> getLinksComlicated() >>>>> File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated >>>>> for file in os.listdir(dir)[1:]: >>>>> WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' >>>>> >>>> >>>> Either the code needs changing so it does a wildcard search or you need >>>> to pass in 'complicated' alone. Seeing those three magic letters P H D >>>> I'll leave you to research the former :) >>>> >>>> -- >>>> My fellow Pythonistas, ask not what our language can do for you, ask >>>> what you can do for our language. >>>> >>>> Mark Lawrence >>> >>> >>> sir what mean by the following errors: >>> Traceback (most recent call last): >>> File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in >>> links = getExpertInfoLinks() >>> File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks >>> fid = open("health-experts.htm","rb") >>> IOError: [Errno 2] No such file or directory: 'health-experts.htm' >> >> File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 149, in >> getForums() >> File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 143, in getForums >> soup = BeautifulSoup(open(file,"r").read()) >> File "C:\Python27\lib\codecs.py", line 878, in open >> file = __builtin__.open(filename, mode, buffering) >> IOError: [Errno 2] No such file or directory: 'links.htm > > Traceback (most recent call last): > File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 106, in > getLinksComlicated() > File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > for file in os.listdir(dir)[1:]: > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > Those errors all mean the same thing. You've got a bug in your code. But since you don't show your code, all we can do is guess. All of those refer to missing files. The first two are missing when you try to open them, so we can be pretty sure they're not in the current directory. In the third case, you're calling listdir() with a directory name with asterisks in it. And those aren't legal in Windows. Fix the directory name, perhaps to "Z:/Your_documents/someplace/else/complicated/" Of course, you probably don't ahve that directory either, but it's more likely than the one you specified. -- DaveA From python at mrabarnett.plus.com Fri Feb 20 20:01:32 2015 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 21 Feb 2015 01:01:32 +0000 Subject: urgent help In-Reply-To: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> Message-ID: <54E7D8EC.7030606@mrabarnett.plus.com> On 2015-02-21 00:14, ms.isma222 at gmail.com wrote: > On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: >> On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: >> >> Would you please give your requests a meaningful subject. >> >> > sir these errors whats mean by it >> > warning (from warnings module): >> > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 >> > br.set_handle_gzip(True) >> > UserWarning: gzip transfer encoding is experimental! >> >> See >> http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do >> >> > >> > Traceback (most recent call last): >> > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in >> > getLinksComlicated() >> > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated >> > for file in os.listdir(dir)[1:]: >> > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' >> > >> >> Either the code needs changing so it does a wildcard search or you need >> to pass in 'complicated' alone. Seeing those three magic letters P H D >> I'll leave you to research the former :) >> > > sir what mean by the following errors: > Traceback (most recent call last): > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in > links = getExpertInfoLinks() > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks > fid = open("health-experts.htm","rb") > IOError: [Errno 2] No such file or directory: 'health-experts.htm' > It means that it can't find that file, either because it doesn't exist, or because it's not in the directory that it's looking in. Notice that it's a relative path (it doesn't start with a drive letter), so it's looking for the file in the current directory. Try opening the file with an absolute path instead. From python at mrabarnett.plus.com Fri Feb 20 20:05:32 2015 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 21 Feb 2015 01:05:32 +0000 Subject: urgent help In-Reply-To: <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> Message-ID: <54E7D9DC.5000404@mrabarnett.plus.com> On 2015-02-21 00:16, ms.isma222 at gmail.com wrote: > On Saturday, February 21, 2015 at 8:14:43 AM UTC+8, ms.is... at gmail.com wrote: >> On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: >> > On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: >> > >> > Would you please give your requests a meaningful subject. >> > >> > > sir these errors whats mean by it >> > > warning (from warnings module): >> > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 >> > > br.set_handle_gzip(True) >> > > UserWarning: gzip transfer encoding is experimental! >> > >> > See >> > http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do >> > >> > > >> > > Traceback (most recent call last): >> > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in >> > > getLinksComlicated() >> > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated >> > > for file in os.listdir(dir)[1:]: >> > > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' >> > > >> > >> > Either the code needs changing so it does a wildcard search or you need >> > to pass in 'complicated' alone. Seeing those three magic letters P H D >> > I'll leave you to research the former :) >> > >> >> sir what mean by the following errors: >> Traceback (most recent call last): >> File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in >> links = getExpertInfoLinks() >> File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks >> fid = open("health-experts.htm","rb") >> IOError: [Errno 2] No such file or directory: 'health-experts.htm' > > File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 149, in > getForums() > File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 143, in getForums > soup = BeautifulSoup(open(file,"r").read()) > File "C:\Python27\lib\codecs.py", line 878, in open > file = __builtin__.open(filename, mode, buffering) > IOError: [Errno 2] No such file or directory: 'links.htm > It looks like the same problem, and the same solution, as the previous one, i.e. try an absolute path instead of a relative path. From python at mrabarnett.plus.com Fri Feb 20 20:08:04 2015 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 21 Feb 2015 01:08:04 +0000 Subject: urgent help In-Reply-To: <73f29253-aeea-4220-b89c-59e61cbfb418@googlegroups.com> References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> <63e4977f-bf58-4c3b-9706-d9e3bbd13fb2@googlegroups.com> <73f29253-aeea-4220-b89c-59e61cbfb418@googlegroups.com> Message-ID: <54E7DA74.5090301@mrabarnett.plus.com> On 2015-02-21 00:20, ms.isma222 at gmail.com wrote: > On Saturday, February 21, 2015 at 8:17:06 AM UTC+8, ms.is... at gmail.com wrote: >> On Saturday, February 21, 2015 at 8:14:43 AM UTC+8, ms.is... at gmail.com wrote: >> > On Friday, February 20, 2015 at 4:41:57 PM UTC+8, Mark Lawrence wrote: >> > > On 20/02/2015 08:24, ismahameed at gcuf.edu.pk wrote: >> > > >> > > Would you please give your requests a meaningful subject. >> > > >> > > > sir these errors whats mean by it >> > > > warning (from warnings module): >> > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 18 >> > > > br.set_handle_gzip(True) >> > > > UserWarning: gzip transfer encoding is experimental! >> > > >> > > See >> > > http://stackoverflow.com/questions/21415450/what-does-mechanize-br-set-handle-gzip-do >> > > >> > > > >> > > > Traceback (most recent call last): >> > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 106, in >> > > > getLinksComlicated() >> > > > File "D:\PHD 1st semester\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated >> > > > for file in os.listdir(dir)[1:]: >> > > > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' >> > > > >> > > >> > > Either the code needs changing so it does a wildcard search or you need >> > > to pass in 'complicated' alone. Seeing those three magic letters P H D >> > > I'll leave you to research the former :) >> > > >> > sir what mean by the following errors: >> > Traceback (most recent call last): >> > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 143, in >> > links = getExpertInfoLinks() >> > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line 119, in getExpertInfoLinks >> > fid = open("health-experts.htm","rb") >> > IOError: [Errno 2] No such file or directory: 'health-experts.htm' >> >> File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 149, in >> getForums() >> File "D:\My Documents\Desktop\scripts\BeOk\getBeOKExperts.py", line 143, in getForums >> soup = BeautifulSoup(open(file,"r").read()) >> File "C:\Python27\lib\codecs.py", line 878, in open >> file = __builtin__.open(filename, mode, buffering) >> IOError: [Errno 2] No such file or directory: 'links.htm > > Traceback (most recent call last): > File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 106, in > getLinksComlicated() > File "D:\My Documents\Desktop\scripts\NetDoctor\getNetDoctor.py", line 80, in getLinksComlicated > for file in os.listdir(dir)[1:]: > WindowsError: [Error 3] The system cannot find the path specified: 'complicated/*.*' > You're giving listdir a wildcarded path. Either give it just the (absolute) path of the directory, or use 'glob' (with an absolute path!) instead. From cs at zip.com.au Fri Feb 20 20:11:57 2015 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 21 Feb 2015 12:11:57 +1100 Subject: subprocess command fails In-Reply-To: References: Message-ID: <20150221011157.GA37597@cskk.homeip.net> On 20Feb2015 15:30, Brad s wrote: >I am trying to execute a subprocess, something done in my script a couple of times. But on the last one, it outputs an error I cannot find the solution to. The exact same command using the same files produced at the command line works just fine. > Hi Brad, I have reordered your post in my quote for readability. It is best to ask your question and give some background first, then list offending output with a description of good output, then the code. You write: >The first command works on the command line: > >dnssec-signzone -e20180330000000 -p -t -g -k Ktest123.com.ksk.key -o test123.com test123.com.external Ktest123.com.zsk.key >Verifying the zone using the following algorithms: RSASHA256. >Zone fully signed: >Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked > ZSKs: 1 active, 0 stand-by, 0 revoked >test123.com.external.signed >Signatures generated: 9 >Signatures retained: 0 >Signatures dropped: 0 >Signatures successfully verified: 0 >Signatures unsuccessfully verified: 0 >Signing time in seconds: 0.010 >Signatures per second: 875.401 >Runtime in seconds: 0.013 Which is excellent. Then you supply your code: >sfmove = subprocess.call(['dnssec-signzone','-e',strftime('%Y%m%d%H', gmtime())+'0000','-p','-t','-g','-k',zcombined+'.ksk.key','-o',dname,dname+'.external',zcombined+'.zsk.key']) I would start by pointing out that this is not identical to your shell command line. FOr example, your shell command line supplied the -e optin as "-e20180330000000", but your python code generates two strings: "-e", "20180330000000". It may not matter, but it is not identically worded and some commands are picky about this kind of thing. For debugging purposes I would do two things: - compute the python command argument list as a separate list and print it out, example: cmdargv = [ 'dnssec-signzone', '-e', strftime('%Y%m%d%H', gmtime())+'0000', '-p', '-t', '-g', '-k', zcombined+'.ksk.key', '-o', dname, dname+'.external', zcombined+'.zsk.key' ] print("command = %r" % (cmdargv,)) sfmove = subprocess.call(cmdargv) Note the use of %r to print the contents of the command clearly. (BTW, "sfmove"? Surely "sign" or something.) - hand run _exactly_ the command printed out by the print call above If you run _exactly_ what your python comman did, you should be able to reproduce the error. Then debug on the command line. Then fix the python code accordingly. >#cmd = "dnssec-signzone','-e',strftime('%Y%m%d%H', gmtime())+'0000','-p','-t','-g','-k','K'+dname+'.ksk.key','-o',dname,dname+'.external','K"+dname+'.zsk.key' >#subprocess.check_call(shlex.split(cmd)) Remark: shlex.split is not a good way to make a command line unless it comand from some input line obtained from a user. Stick with your current "construct a list" approach: more robust. I notice your code removes a bunch of files. Might it remove a necessary file for the command? Cheers, Cameron Simpson From rosuav at gmail.com Fri Feb 20 20:24:26 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Feb 2015 12:24:26 +1100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: <87a908e0zh.fsf@jester.gateway.pace.com> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> Message-ID: On Sat, Feb 21, 2015 at 8:17 AM, Paul Rubin wrote: > Ben Finney writes: >> I don't know of a free-software concurrent RDBMS which can be considered >> lighter than that. (No, MySQL doesn't count; its concurrency is >> *unreliable* and it commonly loses data silently. Don't use MySQL.) > > I thought they fixed MySQL transactions years ago, with the InnoDB > engine. For some reason it's not the default, so you have to turn it on > explicitly: is there more to it than that? Even if you use InnoDB for all of _your_ tables, the system catalog tables will all be MyISAM. So it's possible to lose critical metadata. ChrisA From marfig at gmail.com Fri Feb 20 21:42:20 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Sat, 21 Feb 2015 03:42:20 +0100 Subject: try pattern for database connection with the close method Message-ID: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Hello all, I'm using the following pattern for db access that requires me to close the connection as soon as it is not needed: import sqlite3 as lite try: db = lite.connect('data.db') except lite.DatabaseError: raise OSError('database file corrupt or not found.') else: try: with db: db.execute(sql, parms) except lite.IntegrityError: raise ValueError('invalid data') finally: db.close() Since it's a bit verbose, is there a better way? Note: The user of this API has the whole database functionality abstracted away. Hence the exception channeling in the except clauses. From cfkaran2 at gmail.com Fri Feb 20 21:44:25 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Fri, 20 Feb 2015 21:44:25 -0500 Subject: Design thought for callbacks Message-ID: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Hi all, I'm working on a project that will involve the use of callbacks, and I want to bounce an idea I had off of everyone to make sure I'm not developing a bad idea. Note that this is for python 3.4 code; I don't need to worry about any version of python earlier than that. In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? Thanks, Cem Karan From ckaynor at zindagigames.com Fri Feb 20 21:59:03 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Fri, 20 Feb 2015 18:59:03 -0800 Subject: try pattern for database connection with the close method In-Reply-To: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Fri, Feb 20, 2015 at 6:42 PM, Mario Figueiredo wrote: > import sqlite3 as lite > > try: > db = lite.connect('data.db') > except lite.DatabaseError: > raise OSError('database file corrupt or not found.') > else: > try: > with db: > db.execute(sql, parms) > except lite.IntegrityError: > raise ValueError('invalid data') > finally: > db.close() Two comments: You could remove the "else" statement, as it will work exactly the same with or without it. This will reduce the indentation of the bulk of the code by 1 level. You MIGHT be able to remove the finally...close as the with-statement probably does the same thing. I do not know sqlite3, however, so it may do something different, such as committing, but that would normally be on some transition object you get from a call. Basically, you could probably get the same result with (untested): try: db = lite.connect('data.db') except lite.DatabaseError: raise OSError('database file corrupt or not found.') try: with db: db.execute(sql, parms) except lite.IntegrityError: raise ValueError('invalid data') # You may still need the finally, depending on what the with statement does in sqlite3 - you'd have to check the documentation. Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Fri Feb 20 22:13:19 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 21 Feb 2015 14:13:19 +1100 Subject: 'Lite' Databases References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> Message-ID: <85a9087y9c.fsf_-_@benfinney.id.au> Chris Angelico writes: > Even if you use InnoDB for all of _your_ tables, the system catalog > tables will all be MyISAM. So it's possible to lose critical metadata. In addition, MySQL silently [0] loses data in many common situations. That it loses data so *by default* is what makes MySQL strongly anti-recommended. Having an insane default, with an obscure specific toggle switch to turn off only that particular insanity, should not significantly improve one's estimation of MySQL. That it has a great many such insane defaults, that one must discover one at a time, and many of them result in silent data loss, should be enough to swear off MySQL forever. [0] There are warnings created for many of these data losses. Warnings are inadequate in MySQL, because they are ignored by default, do not interrupt the operation, require specific action to retrieve, and become inaccessible immediately after the operation. This counts as ?silent? in my assessment. -- \ ?I don't know half of you half as well as I should like, and I | `\ like less than half of you half as well as you deserve.? ?Bilbo | _o__) Baggins | Ben Finney From denismfmcmahon at gmail.com Fri Feb 20 22:44:42 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Sat, 21 Feb 2015 03:44:42 +0000 (UTC) Subject: urgent help References: <87a659c8-2b37-4c70-92c1-b2f487eb9ebc@googlegroups.com> Message-ID: On Fri, 20 Feb 2015 16:14:18 -0800, ms.isma222 wrote: > sir what mean by the following errors: > Traceback (most recent call last): > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line > 143, in > links = getExpertInfoLinks() > File "D:\My Documents\Desktop\scripts\WebMD\getWebMDExperts.py", line > 119, in getExpertInfoLinks > fid = open("health-experts.htm","rb") This line tried to open a file called health-experts.htm > IOError: [Errno 2] No such file or directory: 'health-experts.htm' This error says the file does not exist. -- Denis McMahon, denismfmcmahon at gmail.com From bcddd214 at gmail.com Fri Feb 20 23:47:24 2015 From: bcddd214 at gmail.com (Brad s) Date: Fri, 20 Feb 2015 20:47:24 -0800 (PST) Subject: subprocess command fails In-Reply-To: References: Message-ID: <86812014-73ad-466a-95ef-4257390b574b@googlegroups.com> Yes, the program deletes the same files it produces. It looks to see if old stuff is there and if it exist, it deletes the files and creates and entirely new DNS zone with keys and then it is supposed to sign it. The last step is where it is still failing but your code helped me to figure out why. I am signing the zone with the current date/date (which is incorrect but was done for testing purposes. I figured out how to add 2 years to the current data/time but I am unable to strip the formatting once I apply my + 2 years trick. How can I add 2 years to stripped format date/time? # dnssec-signzone -e20150221040000 -p -t -g -k Ktest123.com.ksk.key -o test123.com test123.com.external Ktest123.com.zsk.key dnssec-signzone: fatal: No self-signed KSK DNSKEY found. Supply an active key with the KSK flag set, or use '-P'. # dnssec-signzone -e20160221040000 -p -t -g -k Ktest123.com.ksk.key -o test123.com test123.com.external Ktest123.com.zsk.key Verifying the zone using the following algorithms: RSASHA256. Zone fully signed: Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked ZSKs: 1 active, 0 stand-by, 0 revoked test123.com.external.signed Signatures generated: 9 Signatures retained: 0 Signatures dropped: 0 Signatures successfully verified: 0 Signatures unsuccessfully verified: 0 Signing time in seconds: 0.014 Signatures per second: 631.756 Runtime in seconds: 0.018 current code: # cat makekeys.py #!/usr/bin/python3.4 import subprocess import sys import string import os.path import datetime import shlex from time import gmtime, strftime from subprocess import Popen, PIPE, STDOUT pretime = strftime("%Y%m%d%H", gmtime()) time = datetime.datetime.strptime(pretime,'%Y%m%d%H') print (time) plustime = datetime.timedelta(days=730) timeadd = (time + plustime) str(timeadd) #ndate = datetime.strptime(timeadd, '%Y%m%d%H') #timeadd = timeadd.replace(tzinfo=UTC()) print (timeadd) dname = input("Enter the domain to configure keys for? ") if os.path.exists(dname+".external.signed"): os.remove(dname+".external.signed") #os.remove(dname+".external") os.remove(dname+".ksk.key") os.remove(dname+".zsk.key") os.remove(dname+".ksk.private") os.remove(dname+".zsk.private") fd = open( dname+".external", 'w') fd.write("$TTL 86400\n") fd.write("$ORIGIN "+dname+".\n") fd.write("@ 1D IN SOA yoda.ex-mailer.com. admin@"+dname+".(\n") fd.write(" "+strftime("%Y%m%d%H", gmtime())+"\n") #fd.write(" "+repr(timeadd)+"\n") fd.write(" 3h\n") fd.write(" 1h\n") fd.write(" 1w\n") fd.write(" 1h)\n") fd.write(" IN NS yoda.ex-mailer.com.\n") fd.write(" IN NS r2d2.ex-mailer.com.\n") fd.write(dname+". IN TXT v=spf1 mx a:r2d2.ex-mailer.com -all\n") fd.write(dname+". MX 0 r2d2.ex-mailer.com.\n") fd.write("mail."+dname+". IN A 107.191.60.48\n") fd.write("$include /usr/local/etc/namedb/K"+dname+".zsk.key ; ZSK\n") fd.write("$include /usr/local/etc/namedb/K"+dname+".ksk.key ; KSK\n") fd.close() result = subprocess.check_output(["dnssec-keygen", "-f", "KSK", "-r", "/dev/urandom", "-a", "RSASHA256", "-b", "2048", "-n", "ZONE", dname]) result_utf8 = result.decode("utf-8").strip() mylist = list(result_utf8) print (mylist[0]) listlen= len(mylist) array = list() listlen -= 11 i = 0 while( i < listlen ): #if mylist != '\n' ^ mylist != '': array.insert(i, mylist[i]) i = i + 1 combined = "".join(array) print ('combined') print (combined) fmove = subprocess.call(["mv", result_utf8+".key",combined +".ksk.key"]) fmove = subprocess.call(["mv", result_utf8+".private",combined +".ksk.private"]) zresult = subprocess.check_output(["dnssec-keygen","-r","/dev/urandom","-a","RSASHA256","-b","2048","-n","ZONE", dname]) zresult_utf8 = zresult.decode("utf-8").strip() myzlist = list(zresult_utf8) print (myzlist[0]) zlistlen= len(myzlist) zarray = list() zlistlen -= 11 zi = 0 while( zi # cat makekeys.py > #!/usr/bin/python3.4 > import subprocess > import sys > import string > import os.path > import datetime > import shlex > from time import gmtime, strftime > from subprocess import Popen, PIPE, STDOUT > > pretime = strftime("%Y%m%d%H", gmtime()) > time = datetime.datetime.strptime(pretime,'%Y%m%d%H') > print (time) > plustime = datetime.timedelta(days=730) > timeadd = (time + plustime) > str(timeadd) > #ndate = datetime.strptime(timeadd, '%Y%m%d%H') > #timeadd = timeadd.replace(tzinfo=UTC()) > print (timeadd) > > dname = input("Enter the domain to configure keys for? ") > if os.path.exists(dname+".external.signed"): > os.remove(dname+".external.signed") > #os.remove(dname+".external") > os.remove(dname+".ksk.key") > os.remove(dname+".zsk.key") > os.remove(dname+".ksk.private") > os.remove(dname+".zsk.private") > fd = open( dname+".external", 'w') > fd.write("$TTL 86400\n") > fd.write("$ORIGIN "+dname+".\n") > fd.write("@ 1D IN SOA yoda.ex-mailer.com. admin@"+dname+".(\n") > fd.write(" "+strftime("%Y%m%d%H", gmtime())+"\n") > #fd.write(" "+repr(timeadd)+"\n") > fd.write(" 3h\n") > fd.write(" 1h\n") > fd.write(" 1w\n") > fd.write(" 1h)\n") > fd.write(" IN NS yoda.ex-mailer.com.\n") > fd.write(" IN NS r2d2.ex-mailer.com.\n") > fd.write(dname+". IN TXT v=spf1 mx a:r2d2.ex-mailer.com -all\n") > fd.write(dname+". MX 0 r2d2.ex-mailer.com.\n") > fd.write("mail."+dname+". IN A 107.191.60.48\n") > fd.write("$include /usr/local/etc/namedb/K"+dname+".zsk.key ; ZSK\n") > fd.write("$include /usr/local/etc/namedb/K"+dname+".ksk.key ; KSK\n") > fd.close() > > > result = subprocess.check_output(["dnssec-keygen", "-f", "KSK", "-r", "/dev/urandom", "-a", "RSASHA256", "-b", "2048", "-n", "ZONE", dname]) > result_utf8 = result.decode("utf-8").strip() > mylist = list(result_utf8) > print (mylist[0]) > listlen= len(mylist) > array = list() > listlen -= 11 > i = 0 > while( i < listlen ): > #if mylist != '\n' ^ mylist != '': > array.insert(i, mylist[i]) > i = i + 1 > combined = "".join(array) > print ('combined') > print (combined) > fmove = subprocess.call(["mv", result_utf8+".key",combined +".ksk.key"]) > fmove = subprocess.call(["mv", result_utf8+".private",combined +".ksk.private"]) > > zresult = subprocess.check_output(["dnssec-keygen","-r","/dev/urandom","-a","RSASHA256","-b","2048","-n","ZONE", dname]) > zresult_utf8 = zresult.decode("utf-8").strip() > myzlist = list(zresult_utf8) > print (myzlist[0]) > zlistlen= len(myzlist) > zarray = list() > zlistlen -= 11 > zi = 0 > while( zi zarray.insert(zi, myzlist[zi]) > zi = zi + 1 > zcombined = "".join(zarray) > zfmove = subprocess.call(["mv", zresult_utf8+".key",zcombined+".zsk.key"]) > zfmove = subprocess.call(["mv", zresult_utf8+".private",zcombined+".zsk.private"]) > > sfmove = subprocess.call(['dnssec-signzone','-e',strftime('%Y%m%d%H', gmtime())+'0000','-p','-t','-g','-k',zcombined+'.ksk.key','-o',dname,dname+'.external',zcombined+'.zsk.key']) > #cmd = "dnssec-signzone','-e',strftime('%Y%m%d%H', gmtime())+'0000','-p','-t','-g','-k','K'+dname+'.ksk.key','-o',dname,dname+'.external','K"+dname+'.zsk.key' > #subprocess.check_call(shlex.split(cmd)) > > > > > > > # python3.4 makekeys.py > 2015-02-20 23:00:00 > 2017-02-19 23:00:00 > Enter the domain to configure keys for? test123.com > Generating key pair.........+++ ...............+++ > K > combined > Ktest123.com > Generating key pair...+++ .........................................................+++ > K > dnssec-signzone: fatal: No self-signed KSK DNSKEY found. Supply an active > key with the KSK flag set, or use '-P'. > > I am trying to execute a subprocess, something done in my script a couple of times. But on the last one, it outputs an error I cannot find the solution to. The exact same command using the same files produced at the command line works just fine. > > > > command works on the command line: > > dnssec-signzone -e20180330000000 -p -t -g -k Ktest123.com.ksk.key -o test123.com test123.com.external Ktest123.com.zsk.key > Verifying the zone using the following algorithms: RSASHA256. > Zone fully signed: > Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked > ZSKs: 1 active, 0 stand-by, 0 revoked > test123.com.external.signed > Signatures generated: 9 > Signatures retained: 0 > Signatures dropped: 0 > Signatures successfully verified: 0 > Signatures unsuccessfully verified: 0 > Signing time in seconds: 0.010 > Signatures per second: 875.401 > Runtime in seconds: 0.013 From bcddd214 at gmail.com Sat Feb 21 00:14:57 2015 From: bcddd214 at gmail.com (Brad s) Date: Fri, 20 Feb 2015 21:14:57 -0800 (PST) Subject: subprocess command fails In-Reply-To: References: Message-ID: <9edb2725-c91b-4ae2-b5f4-91a80ed3afe8@googlegroups.com> Time adjustment error: # python3.4 timefix.py 2015022105 2015-02-21 05:00:00 Traceback (most recent call last): File "timefix.py", line 15, in ndate = datetime.datetime.strptime(timeadd, '%Y%m%d%H') TypeError: must be str, not datetime.datetime # cat timefix.py #!/usr/bin/python3.4 import subprocess import sys from datetime import datetime import datetime from time import gmtime, strftime pretime = strftime("%Y%m%d%H", gmtime()) time = datetime.datetime.strptime(pretime,'%Y%m%d%H') print(pretime) print (time) plustime = datetime.timedelta(days=730) timeadd = (time + plustime) str(timeadd) ndate = datetime.datetime.strptime(timeadd, '%Y%m%d%H') #timeadd = timeadd.replace(tzinfo=UTC()) print (timeadd) print (ndate) From rosuav at gmail.com Sat Feb 21 00:39:03 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Feb 2015 16:39:03 +1100 Subject: 'Lite' Databases In-Reply-To: <85a9087y9c.fsf_-_@benfinney.id.au> References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <85a9087y9c.fsf_-_@benfinney.id.au> Message-ID: On Sat, Feb 21, 2015 at 2:13 PM, Ben Finney wrote: > In addition, MySQL silently [0] loses data in many common situations. > > > > > Right, I'd forgotten about silent truncations. Though to be fair, the first link refers to some changes which aren't a problem: > Columns that are part of a PRIMARY KEY are made NOT NULL even if not declared that way. The requirement that a PK consist of non-nullable columns is part of other databases, too, and having the words "PRIMARY KEY" imply "NOT NULL PRIMARY KEY" is fine IMO. Back when I used DB2 version 5, I used to use "ID SMALLINT NOT NULL PRIMARY KEY" at the beginning of most of my table definitions, and all this "silent change" does is allow you to abbreviate that. > Trailing spaces are automatically deleted from ENUM and SET member values when the table is created. Which means they're built on top of CHAR columns, not VARCHAR. Not a huge deal IMO, though definitely something to be aware of. Some of the other issues are more concerning, but these two don't bother me, anyway. The other two links, though? Definitely problems. Silent truncation of data is a pest, and it gets even worse than that. I tried to put together a test-case to see if I could do the classic "break two UTF-16 strings and construct a brand new character out of them" trick, but found something even worse. # -*- encoding: UTF-8 -*- from __future__ import print_function import MySQLdb # If you don't explicitly say charset="UTF8", you get some default that you may # not be able to trust. On my test box, it gave me Latin-1. Plus, you can't say # charset="UTF-8" - you have to omit the hyphen. Not good, but not a gotcha as # you get an immediate exception. con = MySQLdb.connect("localhost","demo","demo","demodb",charset="UTF8") cur = con.cursor() # Minor nastiness: This produces a warning if the table doesn't exist. # So automatically displaying warnings will produce annoying noise. cur.execute("drop table if exists nasty") # Don't forget, "utf8" doesn't mean UTF-8... no, you have to say "utf8mb4"! cur.execute("create table nasty (id smallint primary key, payload char(8) charset utf8mb4 not null)") strings = (u"English: safe", u"????????: safe", u"\U0001F4A9: accurate") cur.execute("insert into nasty values (1, %s), (2, %s), (3, %s)", strings) cur.execute("select payload from nasty order by id") print("Input\t\tOutput") for input, output in zip(strings, cur): print(input,output[0],sep="\t") Your terminal may or may not be able to display U+1F4A9, but it's an accurate description of MySQL's handling of astral characters in this demo. I don't even know what's going on here. Tested on MySQL 5.5 on Debian Wheezy. ChrisA From frank at chagford.com Sat Feb 21 00:41:14 2015 From: frank at chagford.com (Frank Millman) Date: Sat, 21 Feb 2015 07:41:14 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: "Cem Karan" wrote in message news:33677AE8-B2FA-49F9-9304-C8D93784255D at gmail.com... > Hi all, I'm working on a project that will involve the use of callbacks, > and I want to bounce an idea I had off of everyone to make sure I'm not > developing a bad idea. Note that this is for python 3.4 code; I don't > need to worry about any version of python earlier than that. > > In order to inform users that certain bits of state have changed, I > require them to register a callback with my code. The problem is that > when I store these callbacks, it naturally creates a strong reference to > the objects, which means that if they are deleted without unregistering > themselves first, my code will keep the callbacks alive. Since this could > lead to really weird and nasty situations, I would like to store all the > callbacks in a WeakSet > (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That > way, my code isn't the reason why the objects are kept alive, and if they > are no longer alive, they are automatically removed from the WeakSet, > preventing me from accidentally calling them when they are dead. My > question is simple; is this a good design? If not, why not? > Are there any potential 'gotchas' I should be worried about? > I tried something similar a while ago, and I did find a gotcha. The problem lies in this phrase - "if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead." I found that the reference was not removed immediately, but was waiting to be garbage collected. During that window, I could call the callback, which resulted in an error. There may have been a simple workaround. Perhaps someone else can comment. Frank Millman From rosuav at gmail.com Sat Feb 21 00:42:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Feb 2015 16:42:58 +1100 Subject: Design thought for callbacks In-Reply-To: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: > In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? > No, it's not. I would advise using strong references - if the callback is a closure, for instance, you need to hang onto it, because there are unlikely to be any other references to it. If I register a callback with you, I expect it to be called; I expect, in fact, that that *will* keep my object alive. ChrisA From nad at acm.org Sat Feb 21 01:22:06 2015 From: nad at acm.org (Ned Deily) Date: Fri, 20 Feb 2015 22:22:06 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <54E7B0DA.7060303@stoneleaf.us> Message-ID: In article <54E7B0DA.7060303 at stoneleaf.us>, Ethan Furman wrote: > On 02/20/2015 01:17 PM, Paul Rubin wrote: > > > SQLite always seemed bloated (from the embedded NoSQL point of view) and > > fragile to me, and the vendor plays an annoying anti-forking trick, > > which is that the code is released but the developers' test suite is > > secret and proprietary (can be licensed from them for big bucks). > Wow, really? I had just started playing with it, but I don't think I'll > bother now. SQLite is one of the most widely-used, best-documented, best-tested, and well-respected software packages in the world. It is used all over by the place on many different platforms. The code, documentation, and some of the tests are in the public domain, freely usable and forkable by all (though I don't know why anyone would want to fork it). It is true that part of the test suite is only released free to SQLite consortium members. I imagine that is done as an incentive to help finance the on-going development and maintenance of SQLite. http://en.wikipedia.org/wiki/SQLite https://www.sqlite.org/about.html https://www.sqlite.org/testing.html https://www.sqlite.org/copyright.html -- Ned Deily, nad at acm.org From bcddd214 at gmail.com Sat Feb 21 01:41:03 2015 From: bcddd214 at gmail.com (Brad s) Date: Fri, 20 Feb 2015 22:41:03 -0800 (PST) Subject: subprocess command fails In-Reply-To: References: Message-ID: <17160998-e376-438f-8410-4fb8fbe05718@googlegroups.com> fixed with now = datetime.datetime.now() later = now + datetime.timedelta(days=2*365) striplater = later.strftime('%Y%m%d%H') # python3.4 makekeys.py Enter the domain to configure keys for? test1234.com Generating key pair.......................................................................................................................................................................................+++ ......................................................................................................................................................................................+++ K Generating key pair.....+++ .........................................................................+++ K Verifying the zone using the following algorithms: RSASHA256. Zone fully signed: Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked ZSKs: 1 active, 0 stand-by, 0 revoked test1234.com.external.signed Signatures generated: 9 Signatures retained: 0 Signatures dropped: 0 Signatures successfully verified: 0 Signatures unsuccessfully verified: 0 Signing time in seconds: 0.010 Signatures per second: 883.392 Runtime in seconds: 0.013 From no.email at nospam.invalid Sat Feb 21 01:42:53 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 20 Feb 2015 22:42:53 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <54E7B0DA.7060303@stoneleaf.us> Message-ID: <871tljepea.fsf@jester.gateway.pace.com> Ned Deily writes: > (though I don't know why anyone would want to fork it). Same reason lots of people have forked Postgres. Or you might just want to customize it. > I imagine that is done as an incentive to help > finance the on-going development and maintenance of SQLite. It's a pretty unusual and annoying trick that other projects have not felt they had to resort to. > https://www.sqlite.org/testing.html Thanks, that's the page I remember. The TH3 test suite is the interesting one and you can't get it without paying a lot of $$$$$. I guess there is some semantic quibble possible about whether you pay for the test suite, or (as they put it) pay for Consortium membership and then (as a member) get the test suite for free. Do you know the situation with the SQL Logic Test (SLT) also mentioned on that page? From cs at zip.com.au Sat Feb 21 01:47:36 2015 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 21 Feb 2015 17:47:36 +1100 Subject: subprocess command fails In-Reply-To: <9edb2725-c91b-4ae2-b5f4-91a80ed3afe8@googlegroups.com> References: <9edb2725-c91b-4ae2-b5f4-91a80ed3afe8@googlegroups.com> Message-ID: <20150221064736.GA70914@cskk.homeip.net> On 20Feb2015 21:14, Brad s wrote: >Time adjustment error: > ># python3.4 timefix.py >2015022105 >2015-02-21 05:00:00 >Traceback (most recent call last): > File "timefix.py", line 15, in > ndate = datetime.datetime.strptime(timeadd, '%Y%m%d%H') >TypeError: must be str, not datetime.datetime > > > ># cat timefix.py >#!/usr/bin/python3.4 >import subprocess >import sys >from datetime import datetime >import datetime >from time import gmtime, strftime > >pretime = strftime("%Y%m%d%H", gmtime()) >time = datetime.datetime.strptime(pretime,'%Y%m%d%H') >print(pretime) >print (time) >plustime = datetime.timedelta(days=730) >timeadd = (time + plustime) >str(timeadd) >ndate = datetime.datetime.strptime(timeadd, '%Y%m%d%H') >#timeadd = timeadd.replace(tzinfo=UTC()) >print (timeadd) >print (ndate) Well, timeadd is a date object, yes? strptime is for parsing a string for date information, but timeadd is already parsed, as it were. Why are you calling strptime at all? Cheers, Cameron Simpson stomping on roses and strangling kittens leaving no fingerprints cause we wear mittens inflatable prostitutes tied up with strings these are a few of my favorite things - gypsy From nad at acm.org Sat Feb 21 03:17:20 2015 From: nad at acm.org (Ned Deily) Date: Sat, 21 Feb 2015 00:17:20 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <54E7B0DA.7060303@stoneleaf.us> <871tljepea.fsf@jester.gateway.pace.com> Message-ID: In article <871tljepea.fsf at jester.gateway.pace.com>, Paul Rubin wrote: > Ned Deily writes: > > (though I don't know why anyone would want to fork it). > > Same reason lots of people have forked Postgres. Or you might just want > to customize it. Well, for whatever reason one might have, one can: it's public domain software. > > I imagine that is done as an incentive to help > > finance the on-going development and maintenance of SQLite. > It's a pretty unusual and annoying trick that other projects have not > felt they had to resort to. > > > https://www.sqlite.org/testing.html > > Thanks, that's the page I remember. The TH3 test suite is the > interesting one and you can't get it without paying a lot of $$$$$. I > guess there is some semantic quibble possible about whether you pay for > the test suite, or (as they put it) pay for Consortium membership and > then (as a member) get the test suite for free. I don't have any special knowledge of the history or current status of the Consortium but it's not difficult to find information and blog posts about it. It seems like this was an approach Richard Hipp and major users of SQLite took to ensure a sustaining funding model for the project while ensuring its independence. It strikes me as a very reasonable and modest constraint given the immense good that the SQLite project has done for so many other projects over the years. Many other less important projects have foundered for lack of sustained funding. https://www.sqlite.org/consortium.html https://blog.lizardwrangler.com/2008/02/27/the-sqlite-consortium/ > Do you know the situation with the SQL Logic Test (SLT) also mentioned > on that page? No -- Ned Deily, nad at acm.org From no.email at nospam.invalid Sat Feb 21 03:32:09 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 21 Feb 2015 00:32:09 -0800 Subject: 'Lite' Databases (Re: sqlite3 and dates) References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <54E7B0DA.7060303@stoneleaf.us> <871tljepea.fsf@jester.gateway.pace.com> Message-ID: <87wq3bd5rq.fsf@jester.gateway.pace.com> Ned Deily writes: >> Same reason lots of people have forked Postgres. Or you might just want >> to customize it. > Well, for whatever reason one might have, one can: it's public domain > software. Yes, but unlike with most FOSS software, your version has much lower quality assurance than the "official" version because you don't have the big test suite. Even if you don't fork or change the code at all, but you just port it to a new platform or compiler, you really should run the full set of tests, but you can't. > It seems like [the SQLite consortium] was an approach Richard Hipp > and major users of SQLite took to ensure a sustaining funding model > for the project while ensuring its independence. The main benefit of the consortium seems to be very close an intense support from the SQLite core developers, including customization and porting services. Consortium membership apparently starts at $75K a year so I doubt anyone joins just to get the test suite. There is some mention of separate licenses just for the test suite but I didn't see a price tag and I wonder if that generates significant revenue compared to the consortium. > given the immense good that the SQLite project has done for so many > other projects over the years. Maybe I should look into it more. I've never particularly felt the need for it since I've either used client/server databases or else simpler embedded databases like bsddb or even flat files. I actually think Macid/Happstack-state is brilliant but I haven't used it yet. > Many other less important projects have foundered for lack of > sustained funding. Other FOSS databases like Postgres, MySQL, MongoDB, Riak, Cassandra, etc. all seem to be doing fine. GNAT Ada (GCC-based Ada compiler and surrounding ecosystem) is split into free and proprietary components and you have to pay pretty big to get the proprietary parts, but the free parts aren't hobbled in any way (like missing tests) AFAIK. From nulla.epistola at web.de Sat Feb 21 05:44:24 2015 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sat, 21 Feb 2015 11:44:24 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> <4d7a8fd2-144a-4400-b012-1967f6deeded@googlegroups.com> Message-ID: <54E86188.8090307@web.de> Am 20.02.2015 um 15:16 schrieb Dennis Lee Bieber: > > The middle-ground is probably something like the embedded version of > Firebird (pity there has been no updated book -- "The Firebird Book" came > out in 2004, v1.5 while 2.5 is current [Whoops, looks like there /is/ an > update, print-on-demand in three overpriced volumes]) > The second edition as PDF EBook is part of the IBPhoenix DVD (Developer Edition $100). But I can't find out from the website what that DVD contains in its current edition. It used to have Firebird itself and quite a lot of additional software and documentation. Firebird embedded has no network access and multi-user capabilities at all if I read the documentation correctly (http://www.firebirdsql.org/manual/ufb-cs-embedded.html). But "A Firebird embedded server DLL can also be used as a network client" to a regular Firebird server (from the same page). On the other hand triggers and stored procedures should work exactly like the client/server versions (and the procedure language is pretty similar to PostgreSQL, so migration between these two isn't too difficult). So this is a middle ground with other pros and cons than SQLite versus PostgreSQL. From nulla.epistola at web.de Sat Feb 21 05:44:24 2015 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sat, 21 Feb 2015 11:44:24 +0100 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <54e546b5$0$11119$c3e8da3@news.astraweb.com> <54e58ba8$0$11121$c3e8da3@news.astraweb.com> <4d7a8fd2-144a-4400-b012-1967f6deeded@googlegroups.com> Message-ID: <54E86188.8090307@web.de> Am 20.02.2015 um 15:16 schrieb Dennis Lee Bieber: > > The middle-ground is probably something like the embedded version of > Firebird (pity there has been no updated book -- "The Firebird Book" came > out in 2004, v1.5 while 2.5 is current [Whoops, looks like there /is/ an > update, print-on-demand in three overpriced volumes]) > The second edition as PDF EBook is part of the IBPhoenix DVD (Developer Edition $100). But I can't find out from the website what that DVD contains in its current edition. It used to have Firebird itself and quite a lot of additional software and documentation. Firebird embedded has no network access and multi-user capabilities at all if I read the documentation correctly (http://www.firebirdsql.org/manual/ufb-cs-embedded.html). But "A Firebird embedded server DLL can also be used as a network client" to a regular Firebird server (from the same page). On the other hand triggers and stored procedures should work exactly like the client/server versions (and the procedure language is pretty similar to PostgreSQL, so migration between these two isn't too difficult). So this is a middle ground with other pros and cons than SQLite versus PostgreSQL. From gvanem at yahoo.no Sat Feb 21 06:05:46 2015 From: gvanem at yahoo.no (Gisle Vanem) Date: Sat, 21 Feb 2015 12:05:46 +0100 Subject: Python path on windows In-Reply-To: <54E74A09.1060906@davea.name> References: <54E74A09.1060906@davea.name> Message-ID: <54E8668A.7010204@yahoo.no> Dave Angel wrote: > Finally, when py.exe starts, it reads that first (shebang) line, and decides which python interpreter to actually use. py.exe? Do you mean python.exe? Is there a way to make python.exe ignore all Shebang lines in all scripts? I had many generated .py-files with: #!g:\ProgramFiler\Python27\python.EXE After transferring my Python tree to a new Win-8.1 PC, my Python tree was installed under "f:\ProgramFiler\" and my CD-ROM on "g:\" This caused those scripts to access my CD-ROM or pop-up a Win-GUI error dialogue if no CD-ROM was in the drive. Irritating. -- --gv From pfranken85 at gmail.com Sat Feb 21 07:15:50 2015 From: pfranken85 at gmail.com (pfranken85 at gmail.com) Date: Sat, 21 Feb 2015 04:15:50 -0800 (PST) Subject: Best practice: Sharing object between different objects Message-ID: Hello! I have a best-practice question: Imagine I have several hardware devices that I work with on the same I2C bus and I am using the python smbus module for that purpose. The individual devices are sensors, ADC, DAC components. As I said, I would like to derive the corresponding classes from one common class, let's say I2CDevice, so that they can share the same bus connection (I don't want to do a import smbus, ..., self.bus = smbus.SMBus(1) all the time). Of course, I could just pass the the bus reference to each instance, but I am pretty sure that there must be a nicer way to do this. In particular, I imagine the following: It should be possible that I have two classes, ADC_I2C, DAC_I2C which share the same base class. Once I create an instance of ADC_I2C or DAC_I2C it should check whether a bus object exists, if not, it should create one and the other class should be able to use this bus reference as well. Do you get my point? I am pretty sure that such a design pattern should exist, maybe also in the reference of DB connections? Unfortunately I did not find something like this. Any hints are highly appreciated! Thanks! From breamoreboy at yahoo.co.uk Sat Feb 21 07:22:58 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 21 Feb 2015 12:22:58 +0000 Subject: try pattern for database connection with the close method In-Reply-To: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On 21/02/2015 02:42, Mario Figueiredo wrote: > Hello all, > > I'm using the following pattern for db access that requires me to > close the connection as soon as it is not needed: > > import sqlite3 as lite > > try: > db = lite.connect('data.db') > except lite.DatabaseError: > raise OSError('database file corrupt or not found.') > else: > try: > with db: > db.execute(sql, parms) > except lite.IntegrityError: > raise ValueError('invalid data') > finally: > db.close() > > Since it's a bit verbose, is there a better way? > > Note: The user of this API has the whole database functionality > abstracted away. Hence the exception channeling in the except clauses. > Use your context manager at the outer level. import sqlite3 as lite try: with lite.connect('data.db') as db: try: db.execute(sql, parms) except lite.IntegrityError: raise ValueError('invalid data') except lite.DatabaseError: raise OSError('database file corrupt or not found.') -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Sat Feb 21 07:36:14 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 21 Feb 2015 12:36:14 +0000 Subject: Python path on windows In-Reply-To: <54E8668A.7010204@yahoo.no> References: <54E74A09.1060906@davea.name> <54E8668A.7010204@yahoo.no> Message-ID: On 21/02/2015 11:05, Gisle Vanem wrote: > Dave Angel wrote: > >> Finally, when py.exe starts, it reads that first (shebang) line, and >> decides which python interpreter to actually use. > > py.exe? Do you mean python.exe? > py.exe or pyw.exe come with the Python launcher on Windows and work out which version of the interpreter to use. https://www.python.org/dev/peps/pep-0397/ https://docs.python.org/3/using/windows.html#python-launcher-for-windows I think it was first installed with Python 3.3. It's available here https://bitbucket.org/vinay.sajip/pylauncher if you're on an older version. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From cfkaran2 at gmail.com Sat Feb 21 08:13:08 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 21 Feb 2015 08:13:08 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> On Feb 21, 2015, at 12:42 AM, Chris Angelico wrote: > On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: >> In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? >> > > No, it's not. I would advise using strong references - if the callback > is a closure, for instance, you need to hang onto it, because there > are unlikely to be any other references to it. If I register a > callback with you, I expect it to be called; I expect, in fact, that > that *will* keep my object alive. OK, so it would violate the principle of least surprise for you. Interesting. Is this a general pattern in python? That is, callbacks are owned by what they are registered with? In the end, I want to make a library that offers as few surprises to the user as possible, and no matter how I think about callbacks, they are surprising to me. If callbacks are strongly-held, then calling 'del foo' on a callable object may not make it go away, which can lead to weird and nasty situations. Weakly-held callbacks mean that I (as the programmer), know that objects will go away after the next garbage collection (see Frank's earlier message), so I don't get 'dead' callbacks coming back from the grave to haunt me. So, what's the consensus on the list, strongly-held callbacks, or weakly-held ones? Thanks, Cem Karan From cfkaran2 at gmail.com Sat Feb 21 08:14:03 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 21 Feb 2015 08:14:03 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <2D2A18CC-372C-4E96-AC74-3CD17D1C64DB@gmail.com> On Feb 21, 2015, at 12:41 AM, Frank Millman wrote: > > "Cem Karan" wrote in message > news:33677AE8-B2FA-49F9-9304-C8D93784255D at gmail.com... >> Hi all, I'm working on a project that will involve the use of callbacks, >> and I want to bounce an idea I had off of everyone to make sure I'm not >> developing a bad idea. Note that this is for python 3.4 code; I don't >> need to worry about any version of python earlier than that. >> >> In order to inform users that certain bits of state have changed, I >> require them to register a callback with my code. The problem is that >> when I store these callbacks, it naturally creates a strong reference to >> the objects, which means that if they are deleted without unregistering >> themselves first, my code will keep the callbacks alive. Since this could >> lead to really weird and nasty situations, I would like to store all the >> callbacks in a WeakSet >> (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That >> way, my code isn't the reason why the objects are kept alive, and if they >> are no longer alive, they are automatically removed from the WeakSet, >> preventing me from accidentally calling them when they are dead. My >> question is simple; is this a good design? If not, why not? >> Are there any potential 'gotchas' I should be worried about? >> > > I tried something similar a while ago, and I did find a gotcha. > > The problem lies in this phrase - "if they are no longer alive, they are > automatically removed from the WeakSet, preventing me from accidentally > calling them when they are dead." > > I found that the reference was not removed immediately, but was waiting to > be garbage collected. During that window, I could call the callback, which > resulted in an error. > > There may have been a simple workaround. Perhaps someone else can comment. THAT would be one heck of a gotcha! Must have been fun debugging that one! Cem Karan From rosuav at gmail.com Sat Feb 21 08:15:58 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 00:15:58 +1100 Subject: Design thought for callbacks In-Reply-To: <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: On Sun, Feb 22, 2015 at 12:13 AM, Cem Karan wrote: > OK, so it would violate the principle of least surprise for you. Interesting. Is this a general pattern in python? That is, callbacks are owned by what they are registered with? > > In the end, I want to make a library that offers as few surprises to the user as possible, and no matter how I think about callbacks, they are surprising to me. If callbacks are strongly-held, then calling 'del foo' on a callable object may not make it go away, which can lead to weird and nasty situations. Weakly-held callbacks mean that I (as the programmer), know that objects will go away after the next garbage collection (see Frank's earlier message), so I don't get 'dead' callbacks coming back from the grave to haunt me. > > So, what's the consensus on the list, strongly-held callbacks, or weakly-held ones? I don't know about Python specifically, but it's certainly a general pattern in other languages. They most definitely are owned, and it's the only model that makes sense when you use closures (which won't have any other references anywhere). If you're expecting 'del foo' to destroy the object, then you have a bigger problem than callbacks, because that's simply not how Python works. You can't _ever_ assume that deleting something from your local namespace will destroy the object, because there can always be more references. So maybe you need a more clear way of saying "I'm done with this, get rid of it". ChrisA From breamoreboy at yahoo.co.uk Sat Feb 21 08:37:09 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 21 Feb 2015 13:37:09 +0000 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On 21/02/2015 05:41, Frank Millman wrote: > > "Cem Karan" wrote in message > news:33677AE8-B2FA-49F9-9304-C8D93784255D at gmail.com... >> Hi all, I'm working on a project that will involve the use of callbacks, >> and I want to bounce an idea I had off of everyone to make sure I'm not >> developing a bad idea. Note that this is for python 3.4 code; I don't >> need to worry about any version of python earlier than that. >> >> In order to inform users that certain bits of state have changed, I >> require them to register a callback with my code. The problem is that >> when I store these callbacks, it naturally creates a strong reference to >> the objects, which means that if they are deleted without unregistering >> themselves first, my code will keep the callbacks alive. Since this could >> lead to really weird and nasty situations, I would like to store all the >> callbacks in a WeakSet >> (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That >> way, my code isn't the reason why the objects are kept alive, and if they >> are no longer alive, they are automatically removed from the WeakSet, >> preventing me from accidentally calling them when they are dead. My >> question is simple; is this a good design? If not, why not? >> Are there any potential 'gotchas' I should be worried about? >> > > I tried something similar a while ago, and I did find a gotcha. > > The problem lies in this phrase - "if they are no longer alive, they are > automatically removed from the WeakSet, preventing me from accidentally > calling them when they are dead." > > I found that the reference was not removed immediately, but was waiting to > be garbage collected. During that window, I could call the callback, which > resulted in an error. > > There may have been a simple workaround. Perhaps someone else can comment. > > Frank Millman > https://docs.python.org/3/library/gc.html has a collect function. That seems like a simple workaround, but whether or not it classifies as a good solution I'll leave to others, I'm not qualified to say. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From al.basili at gmail.com Sat Feb 21 08:39:09 2015 From: al.basili at gmail.com (alb) Date: 21 Feb 2015 13:39:09 GMT Subject: pypandoc and restructured text References: Message-ID: Hi Wolfgang, Wolfgang Maier wrote: [] > I have pandoc 1.12.2.1 and it recognizes the figure directive just fine > (tested with html output so I cannot say anything about LaTeX). This reminds me that I need to move sooner or later from squeeze to wheezy... Do you know of anyway to install wheezy packages on squeeze? On backports there's nothing more than what comes with the squeeze sources. > I've found an old forum post from 2011: > > https://groups.google.com/forum/#!topic/pandoc-discuss/OmGYDycaMjs > > confirming that the figure directive was not supported at that time. > So, yes, I think upgrading pandoc could help. Thanks a lot for the hint. Maybe I should seriously think about upgrading the whole distro. It's just that Gnome3 really sucks to my taste and I'm not in the mood to look for another Desktop Environment...(maybe I should go back to CDE). Al From fabien.maussion at gmail.com Sat Feb 21 08:46:38 2015 From: fabien.maussion at gmail.com (Fabien) Date: Sat, 21 Feb 2015 14:46:38 +0100 Subject: pypandoc and restructured text References: Message-ID: On 21.02.2015 14:39, alb wrote: > Do you know of anyway to install wheezy packages on squeeze? No need for a new distro. Use virtualenv, this is simple and great: http://simononsoftware.com/virtualenv-tutorial-part-2/ Cheers, Fabien From rosuav at gmail.com Sat Feb 21 08:55:46 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 00:55:46 +1100 Subject: pypandoc and restructured text In-Reply-To: References: Message-ID: On Sun, Feb 22, 2015 at 12:39 AM, alb wrote: > Hi Wolfgang, > > Wolfgang Maier wrote: > [] >> I have pandoc 1.12.2.1 and it recognizes the figure directive just fine >> (tested with html output so I cannot say anything about LaTeX). > > This reminds me that I need to move sooner or later from squeeze to > wheezy... > > Do you know of anyway to install wheezy packages on squeeze? On > backports there's nothing more than what comes with the squeeze sources. > > Thanks a lot for the hint. Maybe I should seriously think about > upgrading the whole distro. It's just that Gnome3 really sucks to my > taste and I'm not in the mood to look for another Desktop > Environment...(maybe I should go back to CDE). > This is a bit off-topic for python-list, being more about Debian than the language, but trying to install just one Wheezy package on Squeeze is probably a bad idea; you'll probably find that it's built against a different version of libc and such, so you'd need to upgrade pretty much your whole system. Your two best options are, IMO: 1) Install pandoc from source, which will then build it against your existing libraries. This may not work (if it depends on a newer version of something), but it should tell you if that's the case. You won't disturb your system in any way. 2) Upgrade the entire OS to either Wheezy or Jessie. Personally, I'm quite happy with Jessie, even though she's not officially stable yet (and have you _seen_ her? Definitely mentally unstable); but whether you go Wheezy or Jessie, grab Xfce rather than Gnome 3. The latest Jessie installers are giving an option on installation as to which desktop environment(s) to install, but if you do it as an upgrade, you may want to explicitly uninstall Gnome and install Xfce, either before or after the upgrade. Happily using a mix of Debian versions here, but all with Xfce. ChrisA From cfkaran2 at gmail.com Sat Feb 21 09:07:42 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 21 Feb 2015 09:07:42 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: On Feb 21, 2015, at 8:15 AM, Chris Angelico wrote: > On Sun, Feb 22, 2015 at 12:13 AM, Cem Karan wrote: >> OK, so it would violate the principle of least surprise for you. Interesting. Is this a general pattern in python? That is, callbacks are owned by what they are registered with? >> >> In the end, I want to make a library that offers as few surprises to the user as possible, and no matter how I think about callbacks, they are surprising to me. If callbacks are strongly-held, then calling 'del foo' on a callable object may not make it go away, which can lead to weird and nasty situations. Weakly-held callbacks mean that I (as the programmer), know that objects will go away after the next garbage collection (see Frank's earlier message), so I don't get 'dead' callbacks coming back from the grave to haunt me. >> >> So, what's the consensus on the list, strongly-held callbacks, or weakly-held ones? > > I don't know about Python specifically, but it's certainly a general > pattern in other languages. They most definitely are owned, and it's > the only model that makes sense when you use closures (which won't > have any other references anywhere). I agree about closures; its the only way they could work. When I was originally thinking about the library, I was trying to include all types of callbacks, including closures and callable objects. The callable objects may pass themselves, or one of their methods to the library, or may do something really weird. Although I just realized that closures may cause another problem. In my code, I expect that many different callbacks can be registered for the same event. Unregistering means you request to be unregistered for the event. How do you do that with a closure? Aren't they anonymous? > If you're expecting 'del foo' to destroy the object, then you have a > bigger problem than callbacks, because that's simply not how Python > works. You can't _ever_ assume that deleting something from your local > namespace will destroy the object, because there can always be more > references. So maybe you need a more clear way of saying "I'm done > with this, get rid of it". Agreed about 'del', and I don't assume that the object goes away at the point. The problem is debugging and determining WHY your object is still around. I know a combination of logging and gc.get_referrers() will probably help you figure out why something is still around, but I'm trying to avoid that headache. I guess the real problem is how this creates cycles in the call graph. User code effectively owns the library code, which via callbacks owns the user code. I have no idea what the best point the cycle is to break it, and not surprise someone down the road. The only idea I have is to redesign the library a little, and make anything that accepts a callback actually be a subclass of collections.abc.Container, or even collections.abc.MutableSet. That makes it very obvious that the object owns the callback, and that you will need to remove your object to unregister it. The only problem is how to handle closures; since they are anonymous, how do you decide which one to remove? Thanks, Cem Karan From davea at davea.name Sat Feb 21 09:10:30 2015 From: davea at davea.name (Dave Angel) Date: Sat, 21 Feb 2015 09:10:30 -0500 Subject: Python path on windows In-Reply-To: <54E8668A.7010204@yahoo.no> References: <54E74A09.1060906@davea.name> <54E8668A.7010204@yahoo.no> Message-ID: <54E891D6.8010506@davea.name> On 02/21/2015 06:05 AM, Gisle Vanem wrote: > Dave Angel wrote: > >> Finally, when py.exe starts, it reads that first (shebang) line, and >> decides which python interpreter to actually use. > > py.exe? Do you mean python.exe? Reread my post, or read Mark's reply to yours. The job of py.exe or pyw.exe is to examine the shebang line and then exec the appropriate python.exe > > Is there a way to make python.exe ignore all Shebang lines > in all scripts? I had many generated .py-files with: > #!g:\ProgramFiler\Python27\python.EXE That's what symlinks are for, on Unix-like systems. Somewhere I read that Windows has had them for a little while now. Those generated files should have pointed to a symlink on your system drive, rather than pointing directly to drive G: But you could always write your own py.exe, which interprets the shebang differently. Or run python.exe yourself, which as far as I know, pays no attention to shebang lines. > > After transferring my Python tree to a new Win-8.1 PC, my Python > tree was installed under "f:\ProgramFiler\" and my CD-ROM on "g:\" > > This caused those scripts to access my CD-ROM or pop-up a Win-GUI > error dialogue if no CD-ROM was in the drive. Irritating. > > -- DaveA From cfkaran2 at gmail.com Sat Feb 21 09:14:39 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 21 Feb 2015 09:14:39 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <66164004-5B92-4435-9782-E2BCCF2E66A6@gmail.com> On Feb 21, 2015, at 8:37 AM, Mark Lawrence wrote: > On 21/02/2015 05:41, Frank Millman wrote: >> >> "Cem Karan" wrote in message >> news:33677AE8-B2FA-49F9-9304-C8D93784255D at gmail.com... >>> Hi all, I'm working on a project that will involve the use of callbacks, >>> and I want to bounce an idea I had off of everyone to make sure I'm not >>> developing a bad idea. Note that this is for python 3.4 code; I don't >>> need to worry about any version of python earlier than that. >>> >>> In order to inform users that certain bits of state have changed, I >>> require them to register a callback with my code. The problem is that >>> when I store these callbacks, it naturally creates a strong reference to >>> the objects, which means that if they are deleted without unregistering >>> themselves first, my code will keep the callbacks alive. Since this could >>> lead to really weird and nasty situations, I would like to store all the >>> callbacks in a WeakSet >>> (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That >>> way, my code isn't the reason why the objects are kept alive, and if they >>> are no longer alive, they are automatically removed from the WeakSet, >>> preventing me from accidentally calling them when they are dead. My >>> question is simple; is this a good design? If not, why not? >>> Are there any potential 'gotchas' I should be worried about? >>> >> >> I tried something similar a while ago, and I did find a gotcha. >> >> The problem lies in this phrase - "if they are no longer alive, they are >> automatically removed from the WeakSet, preventing me from accidentally >> calling them when they are dead." >> >> I found that the reference was not removed immediately, but was waiting to >> be garbage collected. During that window, I could call the callback, which >> resulted in an error. >> >> There may have been a simple workaround. Perhaps someone else can comment. >> >> Frank Millman >> > > https://docs.python.org/3/library/gc.html has a collect function. That seems like a simple workaround, but whether or not it classifies as a good solution I'll leave to others, I'm not qualified to say. Unfortunately, depending on how many objects you have in your object graph, it can slow your code down a fair amount. I think Frank is right about how a WeakSet might be a bad idea in this case. You really need to know if an object is alive or dead, and not some indeterminate state. Thanks, Cem Karan From jeanpierreda at gmail.com Sat Feb 21 09:24:36 2015 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sat, 21 Feb 2015 06:24:36 -0800 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On Fri, Feb 20, 2015 at 9:42 PM, Chris Angelico wrote: > No, it's not. I would advise using strong references - if the callback > is a closure, for instance, you need to hang onto it, because there > are unlikely to be any other references to it. If I register a > callback with you, I expect it to be called; I expect, in fact, that > that *will* keep my object alive. For that matter, if the callback is a method, you need to hang onto it, because method wrappers are generated on demand, so the method would be removed from the valid callbacks instantly. Weak references for callbacks are broken. -- Devin From davea at davea.name Sat Feb 21 09:28:24 2015 From: davea at davea.name (Dave Angel) Date: Sat, 21 Feb 2015 09:28:24 -0500 Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: <54E89608.7090007@davea.name> On 02/21/2015 07:15 AM, pfranken85 at gmail.com wrote: > Hello! > > I have a best-practice question: Imagine I have several hardware devices that I work with on the same I2C bus and I am using the python smbus module for that purpose. The individual devices are sensors, ADC, DAC components. As I said, I would like to derive the corresponding classes from one common class, let's say I2CDevice, so that they can share the same bus connection (I don't want to do a import smbus, ..., self.bus = smbus.SMBus(1) all the time). Of course, I could just pass the the bus reference to each instance, but I am pretty sure that there must be a nicer way to do this. > > In particular, I imagine the following: It should be possible that I have two classes, ADC_I2C, DAC_I2C which share the same base class. Once I create an instance of ADC_I2C or DAC_I2C it should check whether a bus object exists, if not, it should create one and the other class should be able to use this bus reference as well. Do you get my point? I am pretty sure that such a design pattern should exist, maybe also in the reference of DB connections? Unfortunately I did not find something like this. > I think you should write the code (or at least a skeleton version). I suspect the answer will become obvious to you, once you're not worrying about design patterns, or being java compatible. Regardless of whether you inherit from a common base class, both classes can have an attribute with a "bus object" in it. The question becomes who initializes it, and who decides to reuse the "same one". Your wording doesn't make that the least bit clear to me; maybe your sample code will. Many times when you'd have a common base class in Java or C++, you just have to have a common member or two in Python. On the other hand, once you start implementing, maybe it'll become obvious that there's enough shared code for a common base class. The first question is whether an ADC "is-a" bus, or "has-a" bus. Stated that way it sounds like you should have an attribute in ADC which is a bus object. > Any hints are highly appreciated! > > Thanks! > -- DaveA From rosuav at gmail.com Sat Feb 21 09:36:35 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 01:36:35 +1100 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: On Sun, Feb 22, 2015 at 1:07 AM, Cem Karan wrote: > I agree about closures; its the only way they could work. When I was originally thinking about the library, I was trying to include all types of callbacks, including closures and callable objects. The callable objects may pass themselves, or one of their methods to the library, or may do something really weird. > > Although I just realized that closures may cause another problem. In my code, I expect that many different callbacks can be registered for the same event. Unregistering means you request to be unregistered for the event. How do you do that with a closure? Aren't they anonymous? > They're objects, same as any other, so the caller can hang onto a reference and then say "now remove this one". Simple example: callbacks = [] def register_callback(f): callbacks.append(f) def unregister_callback(f): callbacks.remove(f) def do_callbacks(): for f in callbacks: f() def make_callback(i): def inner(): print("Callback! %d"%i) register_callback(inner) return inner make_callback(5) remove_me = make_callback(6) make_callback(7) unregister_callback(remove_me) do_callbacks() The other option is for your callback registration to return some kind of identifier, which can later be used to unregister the callback. This is a good way of avoiding reference cycles (the ID could be a simple integer - maybe the length of the list prior to the new callback being appended, and then the unregistration process is simply "callbacks[id] = None", and you skip the Nones when iterating), and even allows you to register the exact same function more than once, for what that's worth. When I do GUI programming, this is usually how things work. For instance, I use GTK2 (though usually with Pike rather than Python), and I can connect a signal to a callback function. Any given signal could have multiple callbacks attached to it, so it's similar to your case. I frequently depend on the GTK engine retaining a reference to my function (and thus to any data it requires), as I tend not to hang onto any inner objects that don't need retention. Once the parent object is destroyed, all its callbacks get dereferenced. Consider this simplified form: def popup_window(): w = Window() # Add layout, info, whatever it takes btn = Button("Close") w.add(btn) # actually it'd be added to a layout btn.signal_connect("clicked", lambda *args: w.destroy()) The GUI back end will hang onto a reference to the window, because it's currently on screen; to the button, because it's attached to the window; and to my function, because it's connected to a button signal. Then when you click the button, the window gets destroyed, which destroys the button, which unregisters all its callbacks. At that point, there are no refs to the function, so it can get disposed of. That button function was the last external reference to the window, and now that it's not on screen, its Python object can also be disposed of, as can the button inside. So it'll all clean up fairly nicely; as long as the callback gets explicitly deregistered, that's the end of everything. ChrisA From rosuav at gmail.com Sat Feb 21 09:40:29 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 01:40:29 +1100 Subject: Python path on windows In-Reply-To: <54E891D6.8010506@davea.name> References: <54E74A09.1060906@davea.name> <54E8668A.7010204@yahoo.no> <54E891D6.8010506@davea.name> Message-ID: On Sun, Feb 22, 2015 at 1:10 AM, Dave Angel wrote: > But you could always write your own py.exe, which interprets the shebang > differently. > > Or run python.exe yourself, which as far as I know, pays no attention to > shebang lines. Or, my preferred solution: Fix the shebangs :) It's usually not hard to put together a script that zips through your hard drive and fixed them... or else you just fix 'em when you notice 'em, which means your shebangs become a kind of historic marker. I used to have drive letters all over my code, and I could track a program's history through them: this used to be on D:, then it moved to E:, etc. Discovering something that still said "D:\eddie" was like digging up a Second World War bullet casing. ChrisA From turiddociccuddu at yahoo.com Sat Feb 21 09:56:28 2015 From: turiddociccuddu at yahoo.com (ME MO CHE OTTO) Date: Sat, 21 Feb 2015 06:56:28 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: <75e1851a-a54d-4913-81db-b4c0adcd56b4@googlegroups.com> -- redacted -- From turiddociccuddu at yahoo.com Sat Feb 21 09:57:22 2015 From: turiddociccuddu at yahoo.com (ME MO CHE OTTO) Date: Sat, 21 Feb 2015 06:57:22 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <6e03e0d4-0ea4-4a51-90f1-94abfb6dd785@googlegroups.com> -- redacted -- From turiddociccuddu at yahoo.com Sat Feb 21 09:57:47 2015 From: turiddociccuddu at yahoo.com (ME MO CHE OTTO) Date: Sat, 21 Feb 2015 06:57:47 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From turiddociccuddu at yahoo.com Sat Feb 21 10:00:39 2015 From: turiddociccuddu at yahoo.com (ME MO CHE OTTO) Date: Sat, 21 Feb 2015 07:00:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: <747a1889-2bd4-4a9c-93e7-155fd1e4b876@googlegroups.com> -- redacted -- From ian.g.kelly at gmail.com Sat Feb 21 10:16:28 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 21 Feb 2015 08:16:28 -0700 Subject: try pattern for database connection with the close method In-Reply-To: References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sat, Feb 21, 2015 at 5:22 AM, Mark Lawrence wrote: > On 21/02/2015 02:42, Mario Figueiredo wrote: >> >> Hello all, >> >> I'm using the following pattern for db access that requires me to >> close the connection as soon as it is not needed: >> >> import sqlite3 as lite >> >> try: >> db = lite.connect('data.db') >> except lite.DatabaseError: >> raise OSError('database file corrupt or not found.') >> else: >> try: >> with db: >> db.execute(sql, parms) >> except lite.IntegrityError: >> raise ValueError('invalid data') >> finally: >> db.close() >> >> Since it's a bit verbose, is there a better way? >> >> Note: The user of this API has the whole database functionality >> abstracted away. Hence the exception channeling in the except clauses. >> > > Use your context manager at the outer level. > > import sqlite3 as lite > > try: > with lite.connect('data.db') as db: > try: > db.execute(sql, parms) > except lite.IntegrityError: > raise ValueError('invalid data') > except lite.DatabaseError: > raise OSError('database file corrupt or not found.') This could result in the OSError being misleadingly raised due to some DatabaseError raised by the execute rather than the connect. From __peter__ at web.de Sat Feb 21 10:22:36 2015 From: __peter__ at web.de (Peter Otten) Date: Sat, 21 Feb 2015 16:22:36 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: Mario Figueiredo wrote: > Hello all, > > I'm using the following pattern for db access that requires me to > close the connection as soon as it is not needed: > > import sqlite3 as lite > > try: > db = lite.connect('data.db') > except lite.DatabaseError: > raise OSError('database file corrupt or not found.') > else: > try: > with db: > db.execute(sql, parms) > except lite.IntegrityError: > raise ValueError('invalid data') > finally: > db.close() > > Since it's a bit verbose, Why would you care about a few lines? You don't repeat them, do you? Put the code into a function or a context manager and invoke it with >>> my_execute(sql, parms) or >>> with my_db() as db: ... db.execute(sql, parms) > is there a better way? > > Note: The user of this API has the whole database functionality > abstracted away. Hence the exception channeling in the except clauses. db.execute() may trigger other sqlite-related errors including DatabaseError: >>> import sqlite3 >>> db = sqlite3.connect("/dev/full") >>> db.execute("create table foo (bar, baz);") Traceback (most recent call last): File "", line 1, in sqlite3.OperationalError: unable to open database file >>> import os >>> with open("data.db", "wb") as f: ... f.write(os.urandom(1024)) # put random bytes into data.db ... # chances of creating a valid db ... # left as an exercise ;) ... 1024 >>> db = sqlite3.connect("data.db") >>> db.execute("create table foo (bar, baz);") Traceback (most recent call last): File "", line 1, in sqlite3.DatabaseError: file is encrypted or is not a database If you want to catch these, too: @contextlib.contextmanager def my_db(): db = None try: db = sqlite3.connect("data.db") with db: yield db # db.execute() if you don't buy # into the contextmanager idea except sqlite3.IntegrityError: raise ValueError except sqlite3.DatabaseError: raise OSError except sqlite3.Error: raise WhateverYouNeed finally: if db is not None: db.close() with my_db() as db: db.execute("select * from sqlite_master;") From __peter__ at web.de Sat Feb 21 10:27:17 2015 From: __peter__ at web.de (Peter Otten) Date: Sat, 21 Feb 2015 16:27:17 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: Ian Kelly wrote: > On Sat, Feb 21, 2015 at 5:22 AM, Mark Lawrence > wrote: >> On 21/02/2015 02:42, Mario Figueiredo wrote: >>> >>> Hello all, >>> >>> I'm using the following pattern for db access that requires me to >>> close the connection as soon as it is not needed: >>> >>> import sqlite3 as lite >>> >>> try: >>> db = lite.connect('data.db') >>> except lite.DatabaseError: >>> raise OSError('database file corrupt or not found.') >>> else: >>> try: >>> with db: >>> db.execute(sql, parms) >>> except lite.IntegrityError: >>> raise ValueError('invalid data') >>> finally: >>> db.close() >>> >>> Since it's a bit verbose, is there a better way? >>> >>> Note: The user of this API has the whole database functionality >>> abstracted away. Hence the exception channeling in the except clauses. >>> >> >> Use your context manager at the outer level. >> >> import sqlite3 as lite >> >> try: >> with lite.connect('data.db') as db: >> try: >> db.execute(sql, parms) >> except lite.IntegrityError: >> raise ValueError('invalid data') >> except lite.DatabaseError: >> raise OSError('database file corrupt or not found.') > > This could result in the OSError being misleadingly raised due to some > DatabaseError raised by the execute rather than the connect. The OP probably wants to catch these DatabaseErrors, too. Also, the chance of a misleading traceback has been greatly reduced with the advent of chained exceptions. From cfkaran2 at gmail.com Sat Feb 21 10:45:22 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 21 Feb 2015 10:45:22 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: On Feb 21, 2015, at 9:36 AM, Chris Angelico wrote: > On Sun, Feb 22, 2015 at 1:07 AM, Cem Karan wrote: >> I agree about closures; its the only way they could work. When I was originally thinking about the library, I was trying to include all types of callbacks, including closures and callable objects. The callable objects may pass themselves, or one of their methods to the library, or may do something really weird. >> >> Although I just realized that closures may cause another problem. In my code, I expect that many different callbacks can be registered for the same event. Unregistering means you request to be unregistered for the event. How do you do that with a closure? Aren't they anonymous? >> > > They're objects, same as any other, so the caller can hang onto a > reference and then say "now remove this one". Simple example: > > callbacks = [] > def register_callback(f): callbacks.append(f) > def unregister_callback(f): callbacks.remove(f) > def do_callbacks(): > for f in callbacks: > f() > > def make_callback(i): > def inner(): > print("Callback! %d"%i) > register_callback(inner) > return inner > > make_callback(5) > remove_me = make_callback(6) > make_callback(7) > unregister_callback(remove_me) > do_callbacks() Yeah, that's pretty much what I thought you'd have to do, which kind of defeats the purpose of closures (fire-and-forget things). BUT it does answer my question, so no complaints about it! So, either you keep a reference to your own closure, which means that the library doesn't really need to, or the library keeps hold of it for you, in which case you don't have a reasonable way of removing it. > The other option is for your callback registration to return some kind > of identifier, which can later be used to unregister the callback. > This is a good way of avoiding reference cycles (the ID could be a > simple integer - maybe the length of the list prior to the new > callback being appended, and then the unregistration process is simply > "callbacks[id] = None", and you skip the Nones when iterating), and > even allows you to register the exact same function more than once, > for what that's worth. That would work. In the cases where someone might register & unregister many callbacks, you might use UUIDs as keys instead (avoids the ABA problem). > When I do GUI programming, this is usually how things work. For > instance, I use GTK2 (though usually with Pike rather than Python), > and I can connect a signal to a callback function. Any given signal > could have multiple callbacks attached to it, so it's similar to your > case. I frequently depend on the GTK engine retaining a reference to > my function (and thus to any data it requires), as I tend not to hang > onto any inner objects that don't need retention. Once the parent > object is destroyed, all its callbacks get dereferenced. Consider this > simplified form: > > def popup_window(): > w = Window() > # Add layout, info, whatever it takes > btn = Button("Close") > w.add(btn) # actually it'd be added to a layout > btn.signal_connect("clicked", lambda *args: w.destroy()) > > The GUI back end will hang onto a reference to the window, because > it's currently on screen; to the button, because it's attached to the > window; and to my function, because it's connected to a button signal. > Then when you click the button, the window gets destroyed, which > destroys the button, which unregisters all its callbacks. At that > point, there are no refs to the function, so it can get disposed of. > That button function was the last external reference to the window, > and now that it's not on screen, its Python object can also be > disposed of, as can the button inside. So it'll all clean up fairly > nicely; as long as the callback gets explicitly deregistered, that's > the end of everything. OK, so if I'm reading your code correctly, you're breaking the cycle in your object graph by making the GUI the owner of the callback, correct? No other chunk of code has a reference to the callback, correct? Thanks, Cem Karan From ian.g.kelly at gmail.com Sat Feb 21 10:50:08 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 21 Feb 2015 08:50:08 -0700 Subject: try pattern for database connection with the close method In-Reply-To: References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sat, Feb 21, 2015 at 8:27 AM, Peter Otten <__peter__ at web.de> wrote: > Ian Kelly wrote: > >> On Sat, Feb 21, 2015 at 5:22 AM, Mark Lawrence >> wrote: >>> try: >>> with lite.connect('data.db') as db: >>> try: >>> db.execute(sql, parms) >>> except lite.IntegrityError: >>> raise ValueError('invalid data') >>> except lite.DatabaseError: >>> raise OSError('database file corrupt or not found.') >> >> This could result in the OSError being misleadingly raised due to some >> DatabaseError raised by the execute rather than the connect. > > The OP probably wants to catch these DatabaseErrors, too. Also, the chance > of a misleading traceback has been greatly reduced with the advent of > chained exceptions. > Yes, but the point is that OSError is probably inappropriate in that case. From rosuav at gmail.com Sat Feb 21 10:55:13 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 02:55:13 +1100 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: On Sun, Feb 22, 2015 at 2:45 AM, Cem Karan wrote: > OK, so if I'm reading your code correctly, you're breaking the cycle in your object graph by making the GUI the owner of the callback, correct? No other chunk of code has a reference to the callback, correct? Correct. The GUI engine ultimately owns everything. Of course, this is a very simple case (imagine a little notification popup; you don't care about it, you don't need to know when it's been closed, the only event on it is "hit Close to destroy the window"), and most usage would have other complications, but it's not uncommon for me to build a GUI program that leaves everything owned by the GUI engine. Everything is done through callbacks. Destroy a window, clean up its callbacks. The main window will have an "on-deletion" callback that terminates the program, perhaps. It's pretty straight-forward. ChrisA From marko at pacujo.net Sat Feb 21 11:03:50 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 21 Feb 2015 18:03:50 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <87zj87i74p.fsf@elektro.pacujo.net> Chris Angelico : > On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: >> In order to inform users that certain bits of state have changed, I >> require them to register a callback with my code. The problem is that >> when I store these callbacks, it naturally creates a strong reference >> to the objects, which means that if they are deleted without >> unregistering themselves first, my code will keep the callbacks >> alive. Since this could lead to really weird and nasty situations, >> [...] > > No, it's not. I would advise using strong references - if the callback > is a closure, for instance, you need to hang onto it, because there > are unlikely to be any other references to it. If I register a > callback with you, I expect it to be called; I expect, in fact, that > that *will* keep my object alive. I use callbacks all the time but haven't had any problems with strong references. I am careful to move my objects to a zombie state after they're done so they can absorb any potential loose callbacks that are lingering in the system. Marko From steve+comp.lang.python at pearwood.info Sat Feb 21 11:15:22 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 03:15:22 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Frank Millman wrote: > I tried something similar a while ago, and I did find a gotcha. > > The problem lies in this phrase - "if they are no longer alive, they are > automatically removed from the WeakSet, preventing me from accidentally > calling them when they are dead." > > I found that the reference was not removed immediately, but was waiting to > be garbage collected. During that window, I could call the callback, which > resulted in an error. I don't understand how this could possibly work. (Or fail to work, as the case may be.) If the callback has been garbage collected, then you cannot call it, because you don't have any references to it and so cannot refer to it in any way. If the callback has *not* been garbage collected, then you can safely call it. You have a reference to the callback, therefore it exists. (If Python ever garbage collects an object that still has references to it, that would be a critical bug, and you would likely get some sort of seg fault). The only thing I can think of is you have a situation where your callback refers to another object, B, via a weak reference. Once all the regular strong references to the callback and B are gone, theoretically you could have a race condition where the callback is waiting to be garbage collected but B has already been garbage collected. If, in that window, you call the callback, *and* if the callback fails to correctly check that the weak reference to B still exists, then you could get a Python exception. The solution is simple: anytime you have a weak reference, you must always check it before you use it. Other than that, I cannot see how calling a function which has *not* yet been garbage collected can fail, just because the only reference still existing is a weak reference. -- Steven From __peter__ at web.de Sat Feb 21 12:02:50 2015 From: __peter__ at web.de (Peter Otten) Date: Sat, 21 Feb 2015 18:02:50 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: Ian Kelly wrote: > On Sat, Feb 21, 2015 at 8:27 AM, Peter Otten <__peter__ at web.de> wrote: >> Ian Kelly wrote: >> >>> On Sat, Feb 21, 2015 at 5:22 AM, Mark Lawrence >>> wrote: >>>> try: >>>> with lite.connect('data.db') as db: >>>> try: >>>> db.execute(sql, parms) >>>> except lite.IntegrityError: >>>> raise ValueError('invalid data') >>>> except lite.DatabaseError: >>>> raise OSError('database file corrupt or not found.') >>> >>> This could result in the OSError being misleadingly raised due to some >>> DatabaseError raised by the execute rather than the connect. >> >> The OP probably wants to catch these DatabaseErrors, too. Also, the >> chance of a misleading traceback has been greatly reduced with the advent >> of chained exceptions. >> > > Yes, but the point is that OSError is probably inappropriate in that case. Perhaps, but the example I gave in my other post: >>> with open("data.db", "wb") as f: ... f.write(os.urandom(1024)) # put random bytes into data.db >>> db = sqlite3.connect("data.db") >>> db.execute("create table foo (bar, baz);") Traceback (most recent call last): File "", line 1, in sqlite3.DatabaseError: file is encrypted or is not a database matches the "database file corrupt" part of the error message provided by the OP. From marko at pacujo.net Sat Feb 21 12:08:11 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 21 Feb 2015 19:08:11 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87egpjtcp0.fsf@elektro.pacujo.net> Steven D'Aprano : > Other than that, I cannot see how calling a function which has *not* > yet been garbage collected can fail, just because the only reference > still existing is a weak reference. Maybe the logic of the receiving object isn't prepared for the callback anymore after an intervening event. The problem then, of course, is in the logic and not in the callbacks. Marko From no.email at nospam.invalid Sat Feb 21 12:18:57 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 21 Feb 2015 09:18:57 -0800 Subject: Best practice: Sharing object between different objects References: Message-ID: <87sidzchdq.fsf@jester.gateway.pace.com> pfranken85 at gmail.com writes: > ADC, DAC components. As I said, I would like to derive the > corresponding classes from one common class, let's say I2CDevice, so > that they can share the same bus connection (I don't want to do a > import smbus, ..., self.bus = smbus.SMBus(1) all the time). I don't really understand the question but it sounds like you might want a class variable: import smbus class I2CDevice bus = smbus.SMBus(1) def __init__ (self): ... This puts the "bus" object into the I2CDevice class itself. It is created when the I2CDevice class is defined. Methods wanting to access it can then say do_something_with(I2CDevice.bus) is that what you wanted? From bryan0731 at gmail.com Sat Feb 21 12:21:59 2015 From: bryan0731 at gmail.com (Bryan Duarte) Date: Sat, 21 Feb 2015 10:21:59 -0700 Subject: Accessible tools In-Reply-To: <479CFB67BB2640C09ADB16742A547172@JakesPC> References: <479CFB67BB2640C09ADB16742A547172@JakesPC> Message-ID: <00DB5382-A26A-4157-898E-854C6C14CDF0@gmail.com> Eric, Although I would most likely enjoy the former option I feel the latter would be most appropriate for contacting you. Thanks for getting back to me and explaining some of this. I will contact you off list for sure so as not to fill up the lists mailboxes with this topic. I will say that Jacob is absolutely correct in the fact that those of us who rely on screen readers to interact with our computers have a few things we do, and tend to not do. ? Do type very efficiently ? Do listen to our speech output very fast ? Do interact with the text line, word, and character at a time. ? Do not, use a mouse ? Do not, enjoy reading through several lines of code at a time to locate a specific issue, method, or block of code. ? Do not, have access to debugging tools. This is just a few things we do and do not do typically, but the point is that even though some things are similar there are some things that are quite different. Here is an example. I would not use a voice driven editor because I tend to build out a layout of code before I begin filling in methods, functions, and constructors. For me to do this with voice would require the use of a mouse to place the cursor where I want each line of code to be placed, and since I do not have access to the mouse in that way this would not work well. On the other hand if the program could interpret what you are speaking to build out method signatures and individual lines of code accurately, then there is a good chance that same system could provide a great interface for a screen reader. Thank you again for the response and be expecting an email from me shortly sir. > On Feb 20, 2015, at 11:39 AM, Jacob Kruger wrote: > > Eric, issue is that with screenreaders, we're generally way more into > navigating code and interface character by character/by keyboard, so , yes, > keeping interface relatively simple is a good thing, but, we also would > prefer to primarily keep all interface elements to make use of standard UI > controls, and make sure tab index/order is suitable/relevant at times, etc. > etc. > > As in, I think we'd primarily want to avoid having to use a mouse at all if > possible, but anyway. > > Stay well > > Jacob Kruger > Blind Biker > Skype: BlindZA > "Roger Wilco wants to welcome you...to the space janitor's closet..." > > ----- Original Message ----- From: "Eric S. Johansson" > To: > Sent: Friday, February 20, 2015 7:22 PM > Subject: Re: Accessible tools > > >> >> On 2/19/2015 10:33 AM, Bryan Duarte wrote: >>> Thank you jwi, and Jacob, >>> >>> I took a look at that posting and it seems pretty unique. I am not much interested in the speech driven development, but I am very interested in developing an accessible IDE. >> >> Well you should be because it looks like an aural interface (uses speech instead of keyboards) uses the same kinds of data to present to either a text to speech or speech recognition driven environment. >>> A professor and I have been throwing around the idea of developing a completely text based IDE. There are a lot of reasons this could be beneficial to a blind developer and maybe even some sighted developers who are comfortable in the terminal. The idea would be really just to provide a way of easily navigating blocks of code using some kind of tabular formatting, and being able to collapse blocks of code and hearing from a high level information about the code within. All tools and features would obviously be spoken or output in some kind of audio manor. >> I've been working with another professor working on some of these issues as well. His focus has been mostly blind young adults in India. come up with some pretty cool concepts that looks very usable. The challenge now is to make them work and, quite frankly monetize the effort to pay for the development. >> >> Again, this shows the similarities in functionality used by both speech recognition and text-to-speech. All I care about is text and what I can say. We're now working with constructs such as with-open, argument by number, plaintext symbol names (with bidirectional transform to and from code form), guided construct generation for things like classes, methods, comprehensions etc. >> >> All of these things would be useful to handed programmers as well as a way of accelerating co-creation and editing. Unfortunately, like with disabled people stove piping text-to-speech versus speech recognition, handed developers stovepipe keyboard interfaces and don't really think about what they are trying to do, only how they are doing it. >> >> Yes yes, it's a broadbrush that you can probably slap me with. :-) >>> >>> Oh and before I forget does anyone know how to contact Eric who was developing that accessible speech driven IDE? Thanks >> >> Well, you could try looking in a mirror and speaking my name three times at midnight But you would get better results if you used my non-mailing list email address. esj at eggo.org. >> >> --- eric >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > -- > https://mail.python.org/mailman/listinfo/python-list From steve+comp.lang.python at pearwood.info Sat Feb 21 12:27:50 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 04:27:50 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> Cem Karan wrote: > > On Feb 21, 2015, at 8:15 AM, Chris Angelico wrote: > >> On Sun, Feb 22, 2015 at 12:13 AM, Cem Karan wrote: >>> OK, so it would violate the principle of least surprise for you. >>> Interesting. Is this a general pattern in python? That is, callbacks >>> are owned by what they are registered with? >>> >>> In the end, I want to make a library that offers as few surprises to the >>> user as possible, and no matter how I think about callbacks, they are >>> surprising to me. If callbacks are strongly-held, then calling 'del >>> foo' on a callable object may not make it go away, which can lead to >>> weird and nasty situations. How? The whole point of callbacks is that you hand over responsibility to another piece of code, and then forget about your callback. The library will call it, when and if necessary, and when the library no longer needs your callback, it is free to throw it away. (If I wish the callback to survive beyond the lifetime of your library's use of it, I have to keep a reference to the function.) >>> Weakly-held callbacks mean that I (as the >>> programmer), know that objects will go away after the next garbage >>> collection (see Frank's earlier message), so I don't get 'dead' >>> callbacks coming back from the grave to haunt me. I'm afraid this makes no sense to me. Can you explain, or better still demonstrate, a scenario where "dead callbacks rise from the grave", so to speak? >>> So, what's the consensus on the list, strongly-held callbacks, or >>> weakly-held ones? >> >> I don't know about Python specifically, but it's certainly a general >> pattern in other languages. They most definitely are owned, and it's >> the only model that makes sense when you use closures (which won't >> have any other references anywhere). > > I agree about closures; its the only way they could work. *scratches head* There's nothing special about closures. You can assign them to a name like any other object. def make_closure(): x = 23 def closure(): return x + 1 return closure func = make_closure() Now you can register func as a callback, and de-register it when your done: register(func) unregister(func) Of course, if you thrown away your reference to func, you have no (easy) way of de-registering it. That's no different to any other object which is registered by identity. (Registering functions by name is a bad idea, since multiple functions can have the same name.) As an alternative, your callback registration function might return a ticket for the function: ticket = register(func) del func unregister(ticket) but that strikes me as over-kill. And of course, the simplest ticket is to return the function itself :-) > When I was > originally thinking about the library, I was trying to include all types > of callbacks, including closures and callable objects. The callable > objects may pass themselves, or one of their methods to the library, or > may do something really weird. I don't think they can do anything too weird. They have to pass a callable object. Your library just calls that object. You shouldn't need to care whether it is a function, a method, a type, a callable instance, or something else. You just call it, and when you're done calling it forever, you just throw it away. > Although I just realized that closures may cause another problem. In my > code, I expect that many different callbacks can be registered for the > same event. Unregistering means you request to be unregistered for the > event. How do you do that with a closure? Aren't they anonymous? Not unless you create them using lambda. Using the make_closure function above: py> func = make_closure() py> func.__name__ 'closure' Of course, if you call make_closure twice, both functions will have the same internal name. You can set the function __name__ and __qualname__ to fix that. This is how the functools.wraps decorator works. But that's a red herring. Don't register functions by name! Not all callable objects have names, and those that do, you may have multiple *distinct* callbacks with the same name. There are two reasonable approaches: unregister by identity, or by returning a ticket which uniquely identifies the callback. The user is responsible for keeping track of their own ticket. If I lose it, I can't unregister my callback any more. So sad, sucks to be me. The simplest possible identity-based scheme would be something like this: # don't hate me for using a global variable CALLBACKS = [] def register(func): if func not in CALLBACKS: CALLBACKS.append(func) def unregister(func): try: CALLBACKS.remove(func) except ValueError: pass That's probably a bit too simple, since it won't behave as expected with bound methods. The problem is that bound methods are generated on the fly, so this won't work: register(instance.spam) # later unregister(instance.spam) # a different instance! I would have to do this: bound_method = instance.spam register(bound_method) unregister(bound_method) But a more sophisticated unregister function should work: # Untested def unregister(func): for i, f in enumerate(CALLBACKS): if (f is func) or (isinstance(f, types.MethodType) and f.__wrapped__ is func): del CALLBACKS[i] return The simplest possible ticket-based system is probably something like this: CALLBACKS = {} NEXT_TICKET = 1 def register(func): global NEXT_TICKET ticket = NEXT_TICKET NEXT_TICKET += 1 callbacks[ticket] = func return ticket def unregister(ticket): if ticket in CALLBACKS: del CALLBACKS[ticket] >> If you're expecting 'del foo' to destroy the object, then you have a >> bigger problem than callbacks, because that's simply not how Python >> works. You can't _ever_ assume that deleting something from your local >> namespace will destroy the object, because there can always be more >> references. So maybe you need a more clear way of saying "I'm done >> with this, get rid of it". > > Agreed about 'del', and I don't assume that the object goes away at the > point. The problem is debugging and determining WHY your object is still > around. I know a combination of logging and gc.get_referrers() will > probably help you figure out why something is still around, but I'm trying > to avoid that headache. Why do you care? Surely all your library should care about is whether or not they have a reference to the callback.If they do, they should call it (when appropriate). If they don't, they aren't responsible for it. > I guess the real problem is how this creates cycles in the call graph. > User code effectively owns the library code, which via callbacks owns the > user code. I have no idea what the best point the cycle is to break it, > and not surprise someone down the road. The only idea I have is to > redesign the library a little, and make anything that accepts a callback > actually be a subclass of collections.abc.Container, or even > collections.abc.MutableSet. That makes it very obvious that the object > owns the callback, and that you will need to remove your object to > unregister it. My brain hurts from the complexity of your solution. What is the actual problem you are trying to solve? I would like to see an example of an actual failure before trying to solve a purely theoretical failure mode. If I register something as a callback, I expect that callback will stay alive for as long as the callbacks are needed. If I might want to unregister it, then I have to keep a reference to the function, otherwise how will I know what I am unregistering? # this makes no sense and cannot work register(func) del func # later unregister(func) So if I do that, (1) it won't work; (2) I'll probably get an exception; (3) the solution is "don't do that"; and (4) solving this problem is not YOUR responsibility. When your code is done with the callbacks, you can just remove them, no questions asked. If I still have a reference to the callback, that reference will still be valid no matter what you do. If I don't have a reference to it, presumably that's because I don't need it any more. I can't access the callback anyway. > The only problem is how to handle closures; since they are > anonymous, how do you decide which one to remove? You identify them by identity, or by a ticket, the same as for any other object. -- Steven From bryan0731 at gmail.com Sat Feb 21 12:28:39 2015 From: bryan0731 at gmail.com (Bryan Duarte) Date: Sat, 21 Feb 2015 10:28:39 -0700 Subject: Accessible tools In-Reply-To: <20150219124346.7a7db5c5@bigbox.christie.dr> References: <436DFEA2-6DE7-488C-A6E7-48A67417CDF6@gmail.com> <54E5B61A.903@wielicki.name> <3D552ED0-20E5-4F5F-863D-BCC012733130@gmail.com> <20150219124346.7a7db5c5@bigbox.christie.dr> Message-ID: Tim, I am also on the blind linux list. I do not often post there as I predominately use a Mac and the Unix terminal but I am using Linux Kali on the side for some side tinkering and learning. I would use Linux a lot more if the screen reader was not so robotic... Would you be willing to be included in some accessibility Q&A or to bounce some ideas off of? > On Feb 19, 2015, at 11:43 AM, Tim Chase wrote: > > While not blind, I have an interest in accessibility and answer a > number of questions on the Blinux (Blind Linux Users) mailing list. > > On 2015-02-19 08:33, Bryan Duarte wrote: >> A professor and I have been throwing around the idea of developing >> a completely text based IDE. There are a lot of reasons this could >> be beneficial to a blind developer and maybe even some sighted >> developers who are comfortable in the terminal. The idea would be >> really just to provide a way of easily navigating blocks of code >> using some kind of tabular formatting, and being able to collapse >> blocks of code and hearing from a high level information about the >> code within. All tools and features would obviously be spoken or >> output in some kind of audio manor. > > It would seem that the traditional Unix-as-IDE[1] would serve you well > here. This is my method of choice, and it allows me to pick my > components and combine them. I usually use tmux, though GNU screen > would do too. Within that, I usually have the following: > > - vim to edit my code. Though swap in your favorite, whether > emacs/emacspeak, ed/edbrowse, joe, nano, or whatever. I know that > at least Vim and emacs support "folding" away blocks of code (what > you describe as "collapsing") which I usually prefix with a comment > that would give you a description of the block > > - a command-line (I use bash, some prefer zsh or tcsh or whatever) > for things like version-control, running my code, and file > management (move/copy/delete/rename/link/etc) > > - a Python command-line REPL that allows me to do quick tests on a > line of code as well as well as make extensive use of Python's > built-in dir() and help() commands which are invaluable. > > - when doing web-development (Django in my case), I'll often have the > dev-server running in one pane, and a console browser like > lynx/links/links2/elinks/w3m in another pane so that I can put my > code through its paces > > Another benefit of this is that I can run this on my development > machine, but then SSH into the machine from anywhere, reattach to the > tmux/screen session, and have the same configuration right as I left > it. > > The entire tmux/screen session can be run within an accessible > terminal window (I know that some are more accessible than others), > within a terminal screen-reader session (like yasr, screader, or > emacspeak), or even remoted into via an accessible SSH program on your > platform of choice. > > -tkc > > [1] > http://blog.sanctum.geek.nz/series/unix-as-ide/ > > > > > > -- > https://mail.python.org/mailman/listinfo/python-list From python.list at tim.thechases.com Sat Feb 21 13:29:18 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 21 Feb 2015 12:29:18 -0600 Subject: Accessible tools In-Reply-To: <00DB5382-A26A-4157-898E-854C6C14CDF0@gmail.com> References: <479CFB67BB2640C09ADB16742A547172@JakesPC> <00DB5382-A26A-4157-898E-854C6C14CDF0@gmail.com> Message-ID: <20150221122918.7e4e39eb@bigbox.christie.dr> On 2015-02-21 10:21, Bryan Duarte wrote: > those of us who rely on screen readers to interact with our > computers have a few things we do, and tend to not do. [snip] While my experience has shown most of your items to be true, I'd contend that >? Do not, have access to debugging tools. is mistaken or at least misinformed. For Python, I use the "pdb" module all the time, and it's command-line driven. Combined with a multi-terminal (whether multiple windows, virtual consoles, or a tmux/screen session), I can easily bounce back and forth between a "pdb" debugging session and the source code to make edits. Just to check, I fired up the "yasr" terminal screen-reader, launched tmux (using my quiet config, since it updates information on the screen like the time on a regular basis, making it chatty), and stepped through some Python code, checked variables, and walked up/down the call-stack. I know most other languages have similar functionality such as gdb for C code. -tkc From esj at harvee.org Sat Feb 21 14:27:04 2015 From: esj at harvee.org (Eric S. Johansson) Date: Sat, 21 Feb 2015 14:27:04 -0500 Subject: 'Lite' Databases (Re: sqlite3 and dates) In-Reply-To: References: <4154cc37-0bb0-4bf2-a52c-b728c737357c@googlegroups.com> <54E517B4.4000409@stoneleaf.us> <87a908e0zh.fsf@jester.gateway.pace.com> <54E7B0DA.7060303@stoneleaf.us> Message-ID: <54E8DC08.9070200@harvee.org> On 02/21/2015 01:22 AM, Ned Deily wrote: > SQLite is one of the most widely-used, best-documented, best-tested, > and well-respected software packages in the world. yes but is still sql. there are a couple of small scale not-sql databases that look interesting. problem with them is that the creator seem to subscribe to the build-and-toss-into-the-wild school of development. http://buzhug.sourceforge.net/ http://www.pydblite.net/en/index.html both are useful, both could use multi-writer support, and both need some love from the python world. --- eric From xxxxxxxx at xxxxxx.xxx Sat Feb 21 14:46:26 2015 From: xxxxxxxx at xxxxxx.xxx (TommyVee) Date: Sat, 21 Feb 2015 14:46:26 -0500 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements Message-ID: Start off with sets of elements as follows: 1. A,B,E,F 2. G,H,L,P,Q 3. C,D,E,F 4. E,X,Z 5. L,M,R 6. O,M,Y Note that sets 1, 3 and 4 all have the element 'E' in common, therefore they are "related" and form the following superset: A,B,C,D,E,F,X,Z Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 have element 'M' in common, therefore they form the following superset: G,H,L,M,O,P,Q,R,Y I think you get the point. As long as sets have at least 1 common element, they combine to form a superset. Also "links" (common elements) between sets may go down multiple levels, as described in the second case above (2->5->6). Cycles thankfully, are not possible. BTW, the number of individual sets (and resultant supersets) will be very large. I don't know where to start with this. I thought about some type of recursive algorithm, but I'm not sure. I could figure out the Python implementation easy enough, I'm just stumped on the algorithm itself. Anybody have an idea? Thanks, Tom From joel.goldstick at gmail.com Sat Feb 21 15:06:39 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 21 Feb 2015 15:06:39 -0500 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: On Sat, Feb 21, 2015 at 2:46 PM, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore they > are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 have > element 'M' in common, therefore they form the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. As long as sets have at least 1 common element, > they combine to form a superset. Also "links" (common elements) between > sets may go down multiple levels, as described in the second case above > (2->5->6). Cycles thankfully, are not possible. > > BTW, the number of individual sets (and resultant supersets) will be very > large. > > I don't know where to start with this. I thought about some type of > recursive algorithm, but I'm not sure. I could figure out the Python > implementation easy enough, I'm just stumped on the algorithm itself. > > Anybody have an idea? > start with reading about python sets. If you take the intersection of two sets it will return a set with common elements. If that is empty, they don't pass your test. If you take the union, you get a set with the set values in each. > Thanks, Tom > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From ethan at stoneleaf.us Sat Feb 21 15:11:12 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 21 Feb 2015 12:11:12 -0800 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: <54E8E660.8060108@stoneleaf.us> On 02/21/2015 11:46 AM, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore they are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 have element 'M' in common, therefore they form > the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. As long as sets have at least 1 common element, they combine to form a superset. Also > "links" (common elements) between sets may go down multiple levels, as described in the second case above (2->5->6). > Cycles thankfully, are not possible. > > BTW, the number of individual sets (and resultant supersets) will be very large. > > I don't know where to start with this. I thought about some type of recursive algorithm, but I'm not sure. I could > figure out the Python implementation easy enough, I'm just stumped on the algorithm itself. > > Anybody have an idea? Use a Counter (collections.Counter), add all sets, then keep any keys with a count of 2 or more. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From davea at davea.name Sat Feb 21 15:18:54 2015 From: davea at davea.name (Dave Angel) Date: Sat, 21 Feb 2015 15:18:54 -0500 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: <54E8E82E.3090200@davea.name> On 02/21/2015 02:46 PM, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore > they are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 > have element 'M' in common, therefore they form the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. As long as sets have at least 1 common > element, they combine to form a superset. Also "links" (common > elements) between sets may go down multiple levels, as described in the > second case above (2->5->6). Cycles thankfully, are not possible. > > BTW, the number of individual sets (and resultant supersets) will be > very large. > > I don't know where to start with this. I can't see where you've defined "this" yet. I think you're trying to find all combinations of sets that are "related", and that you're assuming some of them won't be. So perhaps you're trying to build "islands" of related sets, and you want each of those islands to be of maximal size. > I thought about some type of > recursive algorithm, but I'm not sure. I could figure out the Python > implementation easy enough, I'm just stumped on the algorithm itself. > Seems like you can have a doubly-nested loop, where your inner loop collects all the sets that are related to the one in the outer loop. You label each group with an island-number, where the island number is simply the lowest number set that's on the island. It's still not clear what you do with each of those islands, but the "superset" you refer to before is simply the OR of everything on the island. Perhaps you want to show all smaller supersets that are composed of at least two sets of an island. That becomes a fairly simple power-of-two iteration of those sets, where you throw out the case where the index is a power of two (ie. contains only one set). As you say, calculating the union or intersection of the various sets in python is trivial. -- DaveA From orgnut at yahoo.com Sat Feb 21 15:33:31 2015 From: orgnut at yahoo.com (Larry Hudson) Date: Sat, 21 Feb 2015 12:33:31 -0800 Subject: subprocess command fails In-Reply-To: <86812014-73ad-466a-95ef-4257390b574b@googlegroups.com> References: <86812014-73ad-466a-95ef-4257390b574b@googlegroups.com> Message-ID: On 02/20/2015 08:47 PM, Brad s wrote: [...] > print("command = %r" % (cmdargv,)) > sfmove = subprocess.call(cmdargv) > > > > also, is the % to Python what precision is to C++? > No. It is like the % in C's printf(). -=- Larry -=- From subhabangalore at gmail.com Sat Feb 21 15:51:20 2015 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Sat, 21 Feb 2015 12:51:20 -0800 (PST) Subject: How to design a search engine in Python? Message-ID: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> Dear Group, I am trying to build a search engine in Python. To do this, I have read tutorials and working methodologies from web and books like Stanford IR book [ http://www-nlp.stanford.edu/IR-book/]. I know how to design a crawler, I know PostgresSql, I am fluent with PageRank, TF-IDF, Zipf's law, etc. I came to know of Whoosh[https://pypi.python.org/pypi/Whoosh/] But I am looking for a total tutorial how to implement it. If any body may kindly direct me. I heard there are good source codes and prototypes, but I am not getting. Apology if this is not a question of the room. I tried to post as this is a room of Python bigwigs. Regards, Subhabrata. From invalid at invalid.invalid Sat Feb 21 15:57:57 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Sat, 21 Feb 2015 20:57:57 +0000 (UTC) Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On 2015-02-21, Cem Karan wrote: > > On Feb 21, 2015, at 12:42 AM, Chris Angelico wrote: > >> On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: >>> In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? >>> >> >> No, it's not. I would advise using strong references - if the callback >> is a closure, for instance, you need to hang onto it, because there >> are unlikely to be any other references to it. If I register a >> callback with you, I expect it to be called; I expect, in fact, that >> that *will* keep my object alive. > > OK, so it would violate the principle of least surprise for you. And me as well. I would expect to be able to pass a closure as a callback and not have to keep a reference to it. Perhaps that just a leftover from working with other languages (javascript, scheme, etc.). It doesn't matter if it's a string, a float, a callback, a graphic or whatever: if I pass your function/library an object, I expect _you_ to keep track of it until you're done with it. > Interesting. Is this a general pattern in python? That is, > callbacks are owned by what they are registered with? I'm not sure what you mean by "owned" or why it matters that it's a callback: it's an object that was passed to you: you need to hold onto a reference to it until you're done with it, and the polite thing to do is to delete references to it when you're done with it. > So, what's the consensus on the list, strongly-held callbacks, or > weakly-held ones? -- Grant From marko at pacujo.net Sat Feb 21 16:57:00 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 21 Feb 2015 23:57:00 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <87egpjapxv.fsf@elektro.pacujo.net> Grant Edwards : > the polite thing to do is to delete references to it when you're done > with it. I disagree with that recommendation. You should do the natural thing and not care who holds references to who. Marko From breamoreboy at yahoo.co.uk Sat Feb 21 17:40:48 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 21 Feb 2015 22:40:48 +0000 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: On 21/02/2015 19:46, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore > they are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 > have element 'M' in common, therefore they form the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. As long as sets have at least 1 common > element, they combine to form a superset. Also "links" (common > elements) between sets may go down multiple levels, as described in the > second case above (2->5->6). Cycles thankfully, are not possible. > > BTW, the number of individual sets (and resultant supersets) will be > very large. > > I don't know where to start with this. I thought about some type of > recursive algorithm, but I'm not sure. I could figure out the Python > implementation easy enough, I'm just stumped on the algorithm itself. > > Anybody have an idea? > > Thanks, Tom A naive approach but should give you something to think about. from collections import defaultdict sets = ({'A','B','E','F'}, {'G','H','L','P','Q'}, {'C','D','E','F'}, {'E','X','Z'}, {'L','M','R'}, {'O','M','Y'}) d = defaultdict(list) for i, aSet in enumerate(sets): for a in aSet: d[a].append(i) superSets = [] for k in d: if len(d[k]) > 1: superSet = set() for i in d[k]: superSet |= sets[i] superSets.append(superSet) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From al.basili at gmail.com Sat Feb 21 18:20:56 2015 From: al.basili at gmail.com (alb) Date: 21 Feb 2015 23:20:56 GMT Subject: pypandoc and restructured text References: Message-ID: Hi Chris, Chris Angelico wrote: [] >> Thanks a lot for the hint. Maybe I should seriously think about >> upgrading the whole distro. It's just that Gnome3 really sucks to my >> taste and I'm not in the mood to look for another Desktop >> Environment...(maybe I should go back to CDE). >> > > This is a bit off-topic for python-list, being more about Debian than > the language, but trying to install just one Wheezy package on Squeeze > is probably a bad idea; you'll probably find that it's built against a > different version of libc and such, so you'd need to upgrade pretty > much your whole system. I apologize for littering the group, thought somehow many users here might have had a similar issues with pypandoc. [] > 2) Upgrade the entire OS to either Wheezy or Jessie. Personally, I'm > quite happy with Jessie, even though she's not officially stable yet > (and have you _seen_ her? Definitely mentally unstable); but whether > you go Wheezy or Jessie, grab Xfce rather than Gnome 3. The latest > Jessie installers are giving an option on installation as to which > desktop environment(s) to install, but if you do it as an upgrade, you > may want to explicitly uninstall Gnome and install Xfce, either before > or after the upgrade. I finally upgraded! And I'm currently trying out xfce! Thanks again for the suggestions. Al p.s.: now pandoc works as expected. From xxxxxxxx at xxxxxx.xxx Sat Feb 21 19:07:39 2015 From: xxxxxxxx at xxxxxx.xxx (TommyVee) Date: Sat, 21 Feb 2015 19:07:39 -0500 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: "TommyVee" wrote in message news:Bg5Gw.1344030$No4.494335 at fx19.iad... Start off with sets of elements as follows: 1. A,B,E,F 2. G,H,L,P,Q 3. C,D,E,F 4. E,X,Z 5. L,M,R 6. O,M,Y Note that sets 1, 3 and 4 all have the element 'E' in common, therefore they are "related" and form the following superset: A,B,C,D,E,F,X,Z Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 have element 'M' in common, therefore they form the following superset: G,H,L,M,O,P,Q,R,Y I think you get the point. As long as sets have at least 1 common element, they combine to form a superset. Also "links" (common elements) between sets may go down multiple levels, as described in the second case above (2->5->6). Cycles thankfully, are not possible. BTW, the number of individual sets (and resultant supersets) will be very large. I don't know where to start with this. I thought about some type of recursive algorithm, but I'm not sure. I could figure out the Python implementation easy enough, I'm just stumped on the algorithm itself. Anybody have an idea? Thanks, Tom Thanks to all! Not only did you turn around my thinking on this, but I also learned about some new Python collection types. I think I have a way to get this to work now. From steve+comp.lang.python at pearwood.info Sat Feb 21 21:04:11 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 13:04:11 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> Message-ID: <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> Marko Rauhamaa wrote: > Grant Edwards : > >> the polite thing to do is to delete references to it when you're done >> with it. > > I disagree with that recommendation. You should do the natural thing and > not care who holds references to who. I don't understand this. What is "the natural thing" if not to delete references to an object when you are done with it? Normally you just let things go out of scope, but if that won't happen, you have to take active steps, such as calling del or setting the reference to None. -- Steven From rosuav at gmail.com Sat Feb 21 21:11:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 13:11:38 +1100 Subject: Design thought for callbacks In-Reply-To: <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 22, 2015 at 1:04 PM, Steven D'Aprano wrote: > Marko Rauhamaa wrote: > >> Grant Edwards : >> >>> the polite thing to do is to delete references to it when you're done >>> with it. >> >> I disagree with that recommendation. You should do the natural thing and >> not care who holds references to who. > > I don't understand this. What is "the natural thing" if not to delete > references to an object when you are done with it? Normally you just let > things go out of scope, but if that won't happen, you have to take active > steps, such as calling del or setting the reference to None. I think the disagreement here is over the interpretation of "done with it". If you drop all references to a connected socket object, Python can rightly assume that you're done with the file and want to close it; but what if you drop all references to a listening socket that's been configured to call a function whenever someone connects? def client_connected(sock): sock.send("Hello!\r\n") # whatever listener = socket(23, on_accept=client_connected) What should happen if that main socket isn't bound to a name? In my opinion, the fact that it's configured for callback mode should mean that it's kept alive. But it's also understandable to want to treat it as "done", that it can be disposed of. It seems weird to me that you should have to have a name somewhere that you'll never use, though. ChrisA From steve+comp.lang.python at pearwood.info Sat Feb 21 23:02:29 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 15:02:29 +1100 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements References: Message-ID: <54e954d6$0$13013$c3e8da3$5496439d@news.astraweb.com> TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore > they are "related" and form the following superset: > > A,B,C,D,E,F,X,Z Sounds to me like you already have an algorithm in mind. It might not be correct or complete, or it might not be efficient, but you performed some steps in your own mind to generate that superset. Write down those steps and you have an algorithm. Once you have the algorithm, you can then turn it into Python code and start revising it from there. I went back to first principles and drew a set diagram with multiple disjoint sets of overlapping sets, and then thought about the process of *adding* each new set to the set-of-supersets. It's hard to describe, but if you start drawing an empty Venn diagram, then add one of your sets to the Venn diagram at a time, merging that set to whatever superset it belongs to, the process should appear quite obvious. Let me walk through the process with your six sets above, plus one extra. First, I create a list of supersets, initially empty: supersets = [] I look at Set #1, and compare to all the supersets. There aren't any, so I make a copy of Set #1 and add it to the supersets: [{A,B,E,F}] I look at Set #2, and compare it to all the supersets. It doesn't intersect any of them, so I add it as a new superset: [{A,B,E,F}, {G,H,L,P,Q}] I look at Set #3. It intersects the first superset, but not the second, so I merge them. Likewise for Set #4: [{A,B,C,D,E,F}, {G,H,L,P,Q}] [{A,B,C,D,E,F,X,Z}, {G,H,L,P,Q}] Set #5 intersects the second superset, but not the first. Same for Set #6: [{A,B,C,D,E,F,X,Z}, {G,H,L,M,P,Q,R}] [{A,B,C,D,E,F,X,Z}, {G,H,L,M,O,P,Q,R,Y}] Now let me add an extra Set #7, {A, R}, which intersects *both* supersets. First I merge it with both: [{A,B,C,D,E,F,R,X,Z}, {A,G,H,L,M,O,P,Q,R,Y}] but I don't stop there. Then I recursively run through the supersets with the same merging process to get: [{A,B,C,D,E,F,G,H,L,M,O,P,Q,R,X,Y,Z}] which is the single minimal, disjoint superset of all seven sets. def merge(sets): """Merge a collection of sets into a list of minimal, disjoint supersets. """ supersets = [] for a in sets: count = 0 for s in supersets: if a&s: s.update(a) count += 1 if count == 0: supersets.append(a.copy()) elif count > 1: supersets = merge(supersets) return supersets Testing this with your six sets gives: py> sets = [set(s) for s in ("ABEF", "GHLPQ", "CDEF", "EXZ", "LMR", "OMY")] py> for s in merge(sets): ... print(','.join(sorted(s))) ... A,B,C,D,E,F,X,Z G,H,L,M,O,P,Q,R,Y Adding set #7 gives: py> for s in merge(sets + [{'A', 'R'}]): ... print(','.join(sorted(s))) ... A,B,C,D,E,F,G,H,L,M,O,P,Q,R,X,Y,Z -- Steven From steve+comp.lang.python at pearwood.info Sat Feb 21 23:38:11 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 15:38:11 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Sun, Feb 22, 2015 at 1:04 PM, Steven D'Aprano > wrote: >> Marko Rauhamaa wrote: >> >>> Grant Edwards : >>> >>>> the polite thing to do is to delete references to it when you're done >>>> with it. >>> >>> I disagree with that recommendation. You should do the natural thing and >>> not care who holds references to who. >> >> I don't understand this. What is "the natural thing" if not to delete >> references to an object when you are done with it? Normally you just let >> things go out of scope, but if that won't happen, you have to take active >> steps, such as calling del or setting the reference to None. > > I think the disagreement here is over the interpretation of "done with > it". If you drop all references to a connected socket object, Python > can rightly assume that you're done with the file and want to close > it; but what if you drop all references to a listening socket that's > been configured to call a function whenever someone connects? > > def client_connected(sock): > sock.send("Hello!\r\n") > # whatever > > listener = socket(23, on_accept=client_connected) > > What should happen if that main socket isn't bound to a name? In my > opinion, the fact that it's configured for callback mode should mean > that it's kept alive. But it's also understandable to want to treat it > as "done", that it can be disposed of. It seems weird to me that you > should have to have a name somewhere that you'll never use, though. But you are using it. You might not be using it by name, but you are using it via the callback function. What did you expect, that Python should read your mind and somehow intuit that you still care about this socket listener, but not some other socket listener that you are done with? If you have a servant that follows you around everywhere, throwing objects away when you stop using them, then naturally if you stop using something it will be thrown away. What did you expect? You don't have to bind the listener to a name. Any reference will do. You can dump it in a bucket: bucket_of_stuff = [] bucket_of_stuff.append(some_function(a, b, c)) bucket_of_stuff.append(make_web_server()) bucket_of_stuff.append(socket(23, on_accept=client_connected)) So long as your bucket is still alive, the garbage collector won't collect it or its contents. Hypothetically we could have a system in place where you instruct the garbage collector to not collect an object, then drop all references to it: gc.never_collect(socket(23, on_accept=client_connected)) but that's a potential memory leak, because now you have no way of telling the GC to collect it again once you've finished listening. If you never finish listening, or at least not until your application shuts down, it doesn't count as a leak. But in general, if listeners might come and go, but you have no way for them to be garbage collected once they are done, then that's a leak. What's the easiest way for the GC to flag an object as "never collect this"? It can keep a reference to it. Keeping a list of things you want to be kept alive is simple, easy and obvious: # hypothetically inside gc.py _ALIVE = [] def never_collect(obj): _ALIVE.append(obj) but that's so trivial that it's not worth putting it in the gc module. Besides, that means any other code could reach into the gc and remove your listener from the "keep alive list" without your knowledge or permission. It's simpler and safer to just keep it alive yourself: alive = [] alive.append(socket(...)) but of course that's just my bucket of stuff under a different name. Using the idiom "keep objects alive by keeping a reference to them" (e.g. bind them to a name, or stick them in a list which you keep) makes things much simpler. You can trivially flag objects as "collect" or "don't collect" as needed, without having to import the gc module or memorise some obscure API or worry about implementation details ("if I flag an object as 'never collect' *twice*, do I have to unflag it twice to undo it?"). You just use the regular interface to the garbage collector: the existence of a reference, any reference, keeps an object alive. -- Steven From steve+comp.lang.python at pearwood.info Sat Feb 21 23:42:28 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 15:42:28 +1100 Subject: How to design a search engine in Python? References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> Message-ID: <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> subhabangalore at gmail.com wrote: > Dear Group, > > I am trying to build a search engine in Python. How to design a search engine in Python? First, design a search engine. Then, write Python code to implement that search engine. > To do this, I have read tutorials and working methodologies from web and > books like Stanford IR book [ http://www-nlp.stanford.edu/IR-book/]. I > know how to design a crawler, I know PostgresSql, I am fluent with > PageRank, TF-IDF, Zipf's law, etc. I came to know of > Whoosh[https://pypi.python.org/pypi/Whoosh/] How does your search engine work? What does it do? You MUST be able to describe the workings of your search engine in English, or the natural language of your choice. Write out the steps that it must take, the tasks that it must perform. This is your algorithm. Without an algorithm, how do you expect to write code? What will the code do? Once you have designed your search engine algorithm, then *and only then* should you start to write code to implement that algorithm. -- Steven From subhabangalore at gmail.com Sun Feb 22 00:02:34 2015 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Sat, 21 Feb 2015 21:02:34 -0800 (PST) Subject: How to design a search engine in Python? In-Reply-To: <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sunday, February 22, 2015 at 10:12:39 AM UTC+5:30, Steven D'Aprano wrote: > wrote: > > > Dear Group, > > > > I am trying to build a search engine in Python. > > How to design a search engine in Python? > > First, design a search engine. > > Then, write Python code to implement that search engine. > > > > To do this, I have read tutorials and working methodologies from web and > > books like Stanford IR book [ http://www-nlp.stanford.edu/IR-book/]. I > > know how to design a crawler, I know PostgresSql, I am fluent with > > PageRank, TF-IDF, Zipf's law, etc. I came to know of > > Whoosh[https://pypi.python.org/pypi/Whoosh/] > > How does your search engine work? What does it do? > > You MUST be able to describe the workings of your search engine in English, > or the natural language of your choice. Write out the steps that it must > take, the tasks that it must perform. This is your algorithm. Without an > algorithm, how do you expect to write code? What will the code do? > > Once you have designed your search engine algorithm, then *and only then* > should you start to write code to implement that algorithm. > > > > > -- > Steven Dear Sir, Thank you for your suggestion. But I was looking for a small tutorial of algorithm of the whole engine. I would try to check it build individual modules and integrate them. I was getting some in google and youtube, but I tried to consult you as I do not know whether they would be fine. I am trying your way, let me see how much I go. There are so many search algorithms in our popular data structure books, that is not an issue but how a search engine is getting done, I am thinking bit on that. Regards, Subhabrata. From denismfmcmahon at gmail.com Sun Feb 22 00:37:51 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Sun, 22 Feb 2015 05:37:51 +0000 (UTC) Subject: How to design a search engine in Python? References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, 21 Feb 2015 21:02:34 -0800, subhabangalore wrote: > Thank you for your suggestion. But I was looking for a small tutorial of > algorithm of the whole engine. I would try to check it build individual > modules and integrate them. I was getting some in google and youtube, > but I tried to consult you as I do not know whether they would be fine. > I am trying your way, let me see how much I go. There are so many search > algorithms in our popular data structure books, that is not an issue but > how a search engine is getting done, I am thinking bit on that. Presumably a search engine is simply a database of keyword -> result, possibly with some scoring factor. Calculating scoring factor is going to be fun. Then of course result pages might have scoring factors too. What about a search with multiple keywords. Some result pages might match more than one keyword, so you might add their score for each keyword together to get the ranking in that enquiry for that page. But then pages with lots and lots of different keywords might be low scoring, because searchers are looking for content, not pages of keywords. Finally, What special, unique feature is your search engine going to have that makes it better than all the existing ones? -- Denis McMahon, denismfmcmahon at gmail.com From subhabangalore at gmail.com Sun Feb 22 01:07:30 2015 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Sat, 21 Feb 2015 22:07:30 -0800 (PST) Subject: How to design a search engine in Python? In-Reply-To: References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> Message-ID: <0914bd64-6a5b-4014-af6e-f41dcb0c4fba@googlegroups.com> On Sunday, February 22, 2015 at 11:08:47 AM UTC+5:30, Denis McMahon wrote: > On Sat, 21 Feb 2015 21:02:34 -0800, subhabangalore wrote: > > > Thank you for your suggestion. But I was looking for a small tutorial of > > algorithm of the whole engine. I would try to check it build individual > > modules and integrate them. I was getting some in google and youtube, > > but I tried to consult you as I do not know whether they would be fine. > > I am trying your way, let me see how much I go. There are so many search > > algorithms in our popular data structure books, that is not an issue but > > how a search engine is getting done, I am thinking bit on that. > > Presumably a search engine is simply a database of keyword -> result, > possibly with some scoring factor. > > Calculating scoring factor is going to be fun. > > Then of course result pages might have scoring factors too. What about a > search with multiple keywords. Some result pages might match more than > one keyword, so you might add their score for each keyword together to > get the ranking in that enquiry for that page. > > But then pages with lots and lots of different keywords might be low > scoring, because searchers are looking for content, not pages of keywords. > > Finally, What special, unique feature is your search engine going to have > that makes it better than all the existing ones? > > -- > Denis McMahon, Dear Sir, Thank you for your kind suggestion. Let me traverse one by one. My special feature is generally Semantic Search, but I am trying to build a search engine first and then go for semantic I feel that would give me a solid background to work around the problem. Regards, Subhabrata. From rosuav at gmail.com Sun Feb 22 01:21:00 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 17:21:00 +1100 Subject: Design thought for callbacks In-Reply-To: <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 22, 2015 at 3:38 PM, Steven D'Aprano wrote: > But you are using it. You might not be using it by name, but you are using > it via the callback function. What did you expect, that Python should read > your mind and somehow intuit that you still care about this socket > listener, but not some other socket listener that you are done with? > > You don't have to bind the listener to a name. Any reference will do. You > can dump it in a bucket: > > bucket_of_stuff = [] > bucket_of_stuff.append(some_function(a, b, c)) > bucket_of_stuff.append(make_web_server()) > bucket_of_stuff.append(socket(23, on_accept=client_connected)) Sure, and whether it's a name or a list-element reference doesn't matter: it seems wrong to have to stash a thing in a bucket in order to keep its callbacks alive. I expect the callbacks _themselves_ to keep it alive. But I can understand the opposite POV. ChrisA From frank at chagford.com Sun Feb 22 01:44:10 2015 From: frank at chagford.com (Frank Millman) Date: Sun, 22 Feb 2015 08:44:10 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: "Steven D'Aprano" wrote in message news:54e8af1b$0$12976$c3e8da3$5496439d at news.astraweb.com... > Frank Millman wrote: > >> I tried something similar a while ago, and I did find a gotcha. >> >> The problem lies in this phrase - "if they are no longer alive, they are >> automatically removed from the WeakSet, preventing me from accidentally >> calling them when they are dead." >> >> I found that the reference was not removed immediately, but was waiting >> to >> be garbage collected. During that window, I could call the callback, >> which >> resulted in an error. > > I don't understand how this could possibly work. (Or fail to work, as the > case may be.) > > If the callback has been garbage collected, then you cannot call it, > because > you don't have any references to it and so cannot refer to it in any way. > > If the callback has *not* been garbage collected, then you can safely call > it. You have a reference to the callback, therefore it exists. (If Python > ever garbage collects an object that still has references to it, that > would > be a critical bug, and you would likely get some sort of seg fault). > > The only thing I can think of is you have a situation where your callback > refers to another object, B, via a weak reference. Once all the regular > strong references to the callback and B are gone, theoretically you could > have a race condition where the callback is waiting to be garbage > collected > but B has already been garbage collected. If, in that window, you call the > callback, *and* if the callback fails to correctly check that the weak > reference to B still exists, then you could get a Python exception. > > The solution is simple: anytime you have a weak reference, you must always > check it before you use it. > > Other than that, I cannot see how calling a function which has *not* yet > been garbage collected can fail, just because the only reference still > existing is a weak reference. > You are right. I tried to reproduce the problem and I can't. Before describing what I think was happening, I want to clarify something. Most of this thread uses the word 'callback' in the sense of an 'asynchronous' scenario - the caller wants something to happen some time in the future, and then forget about it, but it is important that it does actually happen eventually. That is not what I was doing, and it is not what I thought the OP was asking for. "In order to inform users that certain bits of state have changed, I require them to register a callback with my code." This sounds to me like a pub/sub scenario. When a 'listener' object comes into existence it is passed a reference to a 'controller' object that holds state. It wants to be informed when the state changes, so it registers a callback function with the controller. When the controller detects a change in state, it calls all the callback functions, thereby notifying each listener. When the listener goes out of scope, it is important that it deregisters with the controller. Now back to my scenario. You are right that so long as the controller maintains a reference to the callback function, the listener cannot be garbage collected, and therefore the callback will always succeed. As far as I can remember, I had a situation where the listener used the information to pass the information to a gui on a client. When the listener was no longer required, a close() fiunction was called which cleaned up and closed connections. When the callback was called after the listener was closed, whatever it was trying to do failed (I forget the details). Hope this makes sense. Frank From ian.g.kelly at gmail.com Sun Feb 22 02:05:18 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 22 Feb 2015 00:05:18 -0700 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On Sat, Feb 21, 2015 at 1:57 PM, Grant Edwards wrote: > On 2015-02-21, Cem Karan wrote: >> >> On Feb 21, 2015, at 12:42 AM, Chris Angelico wrote: >> >>> On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: >>>> In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? >>>> >>> >>> No, it's not. I would advise using strong references - if the callback >>> is a closure, for instance, you need to hang onto it, because there >>> are unlikely to be any other references to it. If I register a >>> callback with you, I expect it to be called; I expect, in fact, that >>> that *will* keep my object alive. >> >> OK, so it would violate the principle of least surprise for you. > > And me as well. I would expect to be able to pass a closure as a > callback and not have to keep a reference to it. Perhaps that just a > leftover from working with other languages (javascript, scheme, etc.). > It doesn't matter if it's a string, a float, a callback, a graphic or > whatever: if I pass your function/library an object, I expect _you_ to > keep track of it until you're done with it. > >> Interesting. Is this a general pattern in python? That is, >> callbacks are owned by what they are registered with? > > I'm not sure what you mean by "owned" or why it matters that it's a > callback: it's an object that was passed to you: you need to hold onto > a reference to it until you're done with it, and the polite thing to > do is to delete references to it when you're done with it. > >> So, what's the consensus on the list, strongly-held callbacks, or >> weakly-held ones? Count me in the weak-ref crowd. It may be a nuisance to keep a reference around on the object registering the callback, but it's preferable to the alternative of messing around with disposables in order to ensure that the callback gets cleaned up and doesn't create a memory leak. I would also rather have my code fail by losing a callback reference, which should be relatively easy to spot and diagnose, than to have said memory leak go unnoticed. From marko at pacujo.net Sun Feb 22 02:52:24 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 09:52:24 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87r3tiids7.fsf@elektro.pacujo.net> Steven D'Aprano : > Marko Rauhamaa wrote: >> Grant Edwards : >>> the polite thing to do is to delete references to it when you're done >>> with it. >> >> I disagree with that recommendation. You should do the natural thing and >> not care who holds references to who. > > I don't understand this. What is "the natural thing" if not to delete > references to an object when you are done with it? Normally you just let > things go out of scope, but if that won't happen, you have to take active > steps, such as calling del or setting the reference to None. Obviously, if you are hoarding objects in a collection, you're going to land in trouble. What I mean, though, is that you shouldn't think you need to create object destructors where you routinely set all members to None. Marko From wxjmfauth at gmail.com Sun Feb 22 03:15:51 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Sun, 22 Feb 2015 00:15:51 -0800 (PST) Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: <54e954d6$0$13013$c3e8da3$5496439d@news.astraweb.com> References: <54e954d6$0$13013$c3e8da3$5496439d@news.astraweb.com> Message-ID: <503257ca-a61a-4aaf-be09-826c553b3f31@googlegroups.com> >>> z = [set(s) for s in ("ABEF", "GHLPQ", "CDEF", "EXZ", "LMR", "OMY")] >>> x = set() >>> for e in z: ... x = x.union(e) ... >>> x {'A', 'C', 'B', 'E', 'D', 'G', 'F', 'H', 'M', 'L', 'O', 'Q', 'P', 'R', 'Y', 'X', 'Z'} >>> ''.join(x) 'ACBEDGFHMLOQPRYXZ' >>> From rosuav at gmail.com Sun Feb 22 03:17:55 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 19:17:55 +1100 Subject: Design thought for callbacks In-Reply-To: <87r3tiids7.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> Message-ID: On Sun, Feb 22, 2015 at 6:52 PM, Marko Rauhamaa wrote: > What I mean, though, is that you shouldn't think you need to create > object destructors where you routinely set all members to None. Sure, not *routinely*. It'd be a special case where it's not specifically a destructor, and its job is to break a reference cycle. For instance, you might have a close() method that clears out a bunch of references, which will then allow everything to get cleaned up promptly. Or (a very common case for me) a callback saying "remote end is gone" (eg on a socket) might wipe out the callbacks, thus removing their refloops. ChrisA From marko at pacujo.net Sun Feb 22 03:34:41 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 10:34:41 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> Message-ID: <87mw46ibtq.fsf@elektro.pacujo.net> Chris Angelico : > Or (a very common case for me) a callback saying "remote end is gone" > (eg on a socket) might wipe out the callbacks, thus removing their > refloops. Refloops are not to be worried about, let alone removed. Marko From rosuav at gmail.com Sun Feb 22 03:58:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 19:58:38 +1100 Subject: Design thought for callbacks In-Reply-To: <87mw46ibtq.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> Message-ID: On Sun, Feb 22, 2015 at 7:34 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> Or (a very common case for me) a callback saying "remote end is gone" >> (eg on a socket) might wipe out the callbacks, thus removing their >> refloops. > > Refloops are not to be worried about, let alone removed. Why? They force the use of the much slower cycle-detecting GC, rather than the quick and efficient CPython refcounter. I don't know how other Pythons work, but mark-and-sweep has its own costs, and I don't know of any system that's both prompt and able to detect refloops. Helping it along means your program doesn't waste memory. Why such a blanket statement? ChrisA From lac at openend.se Sun Feb 22 04:12:17 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 10:12:17 +0100 Subject: How to design a search engine in Python? In-Reply-To: Message from subhabangalore@gmail.com of "Sat, 21 Feb 2015 22:07:30 -0800." <0914bd64-6a5b-4014-af6e-f41dcb0c4fba@googlegroups.com> References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> <0914bd64-6a5b-4014-af6e-f41dcb0c4fba@googlegroups.com> Message-ID: <201502220912.t1M9CH95032374@fido.openend.se> In a message of Sat, 21 Feb 2015 22:07:30 -0800, subhabangalore at gmail.com write >Dear Sir, > >Thank you for your kind suggestion. Let me traverse one by one. >My special feature is generally Semantic Search, but I am trying to build >a search engine first and then go for semantic I feel that would give me a solid background to work around the problem. > >Regards, >Subhabrata. You may find the API docs surrounding rdelbru.github.io/SIREn/ of interest then. Laura Creighton From marko at pacujo.net Sun Feb 22 04:14:30 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 11:14:30 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> Message-ID: <87ioeui9zd.fsf@elektro.pacujo.net> Chris Angelico : > On Sun, Feb 22, 2015 at 7:34 PM, Marko Rauhamaa wrote: >> Refloops are not to be worried about, let alone removed. > > Why? Because the whole point of GC-languages is that you should stop worrying about memory. Trying to mastermind and micromanage GC in the application is, pardon my French, an antipattern. > They force the use of the much slower cycle-detecting GC, rather than > the quick and efficient CPython refcounter. Java's Hotspot doesn't bother with refcounters but is much faster than Python. CPython's refcounters are a historical accident that a Python application developer shouldn't even be aware of. > I don't know how other Pythons work, but mark-and-sweep has its own > costs, and I don't know of any system that's both prompt and able to > detect refloops. It's exceedingly difficult (and pointless) to detect cycles in your object structures. Python is going to have to do a GC occasionally anyway. Yes, your worst-case response times are going to suffer, but that's the cost of doing business. > Helping it along means your program doesn't waste memory. Why such a > blanket statement? Because worrying Python programmers with evil spirits (reference loops) leads to awkward coding practices and takes away one of the main advantages of Python as a high-level programming language. Marko From greg.ewing at canterbury.ac.nz Sun Feb 22 05:15:30 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 22 Feb 2015 23:15:30 +1300 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: Frank Millman wrote: > "In order to inform users that certain bits of state have changed, I require > them to register a callback with my code." > > This sounds to me like a pub/sub scenario. When a 'listener' object comes > into existence it is passed a reference to a 'controller' object that holds > state. It wants to be informed when the state changes, so it registers a > callback function with the controller. Perhaps instead of registering a callback function, you should be registering the listener object together with a method name. You can then keep a weak reference to the listener object, since if it is no longer referenced elsewhere, it presumably no longer needs to be notified of anything. -- Greg From rosuav at gmail.com Sun Feb 22 05:21:17 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 21:21:17 +1100 Subject: Design thought for callbacks In-Reply-To: <87ioeui9zd.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> <87ioeui9zd.fsf@elektro.pacujo.net> Message-ID: On Sun, Feb 22, 2015 at 8:14 PM, Marko Rauhamaa wrote: >> Helping it along means your program doesn't waste memory. Why such a >> blanket statement? > > Because worrying Python programmers with evil spirits (reference loops) > leads to awkward coding practices and takes away one of the main > advantages of Python as a high-level programming language. Right, and I suppose that, by extension, we should assume that the Python interpreter can optimize this? def fib(x): if x<2: return x return fib(x-2)+fib(x-1) Just because a computer can, in theory, recognize that this is a pure function, doesn't mean that we can and should depend on that. If you want this to be optimized, you either fix your algorithm or explicitly memoize the function - you don't assume that Python can do it for you. Even when you write in a high level language, you need to understand how computers work. ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 22 05:32:06 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 22 Feb 2015 21:32:06 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54e9b027$0$12992$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Sun, Feb 22, 2015 at 3:38 PM, Steven D'Aprano > wrote: >> But you are using it. You might not be using it by name, but you are >> using it via the callback function. What did you expect, that Python >> should read your mind and somehow intuit that you still care about this >> socket listener, but not some other socket listener that you are done >> with? >> >> You don't have to bind the listener to a name. Any reference will do. You >> can dump it in a bucket: >> >> bucket_of_stuff = [] >> bucket_of_stuff.append(some_function(a, b, c)) >> bucket_of_stuff.append(make_web_server()) >> bucket_of_stuff.append(socket(23, on_accept=client_connected)) > > Sure, and whether it's a name or a list-element reference doesn't > matter: it seems wrong to have to stash a thing in a bucket in order > to keep its callbacks alive. I expect the callbacks _themselves_ to > keep it alive. Why? Do you expect that the Python garbage collector special cases callbacks to keep them alive even when there are no references to them? How would it distinguish a callback from some other function? If I stuff a function in a list: [len] would you expect the presence of the function to keep the list alive when there are no references to the list? Apart from "But I really, really, REALLY want a magical pony that feeds itself and never poops and just disappears when I don't want it around!" wishful-thinking, which I *totally* get, I don't see how you think this is even possible. Maybe I'm missing something, but it seems to me that what you're wishing for is impossible. Perhaps if we had a special "callback" type which was treated as a special case by the garbage collector. But that opens up a big can of worms: how do you close/delete objects which are kept alive by the presence of callbacks if you don't have a reference to either the object or the callback? -- Steven From rosuav at gmail.com Sun Feb 22 06:14:34 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 22:14:34 +1100 Subject: Design thought for callbacks In-Reply-To: <54e9b027$0$12992$c3e8da3$5496439d@news.astraweb.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> <54e9b027$0$12992$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 22, 2015 at 9:32 PM, Steven D'Aprano wrote: > Why? Do you expect that the Python garbage collector special cases callbacks > to keep them alive even when there are no references to them? How would it > distinguish a callback from some other function? No no no. It's the other way around. _Something_ has to be doing those callbacks, and it's that _something_ that should be keeping them alive. The fact that it's a registered callback should itself *be* a reference (and not a weak reference), and should keep it alive. ChrisA From lac at openend.se Sun Feb 22 06:28:32 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 12:28:32 +0100 Subject: Design thought for callbacks In-Reply-To: Message from Marko Rauhamaa of "Sun, 22 Feb 2015 11:14:30 +0200." <87ioeui9zd.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> <87ioeui9zd.fsf@elektro.pacujo.net> Message-ID: <201502221128.t1MBSWHm028254@fido.openend.se> somebody, I got confused with the indent level wrote: >> They force the use of the much slower cycle-detecting GC, rather than >> the quick and efficient CPython refcounter. Somebody has misunderstood something here. When it comes to efficient garbage collectors, refcounting is a turtle. The CPython one is no exception. Ref counting, however, is fairly easy to write. But when the PyPy project first replaced its refcounting gc with its very first and therefore not very efficient at all nursery gc ... that was the very first time when a bunch of python programs ran faster on pypy than on CPython. This was before pypy had a JIT. And today the pypy channel is full of people who want to link their C extension into some Python code running on PyPy, and who find that their C extension slows things down. There are lots of reasons for this, but one of the most common problems is 'this C extension is faking refcounting. All of this is wasted effort for PyPy and usually makes the thing unJITable as well.' Many of these people rewrite their C extension as pure Python and find that then, with PyPy, they get the speed improvements they were looking for. So: two points. One reason you might not want to rely on ref counting, because you expect your code to run under PyPy one day. and If you are interested in manipulating garbage collection -- especially if this is for your own pleasure and enjoyment, a worthy goal in my books -- you could do a lot worse than write your own gc in RPython for PyPy. The gc code is not mixed in with all of the other VM stuff, so a gc is small, and you don't have to worry about clobbering anything else while you are working. So it is great for experimenting, which was the whole point. Hacking gcs is fun! :) Laura From fabien.maussion at gmail.com Sun Feb 22 06:59:49 2015 From: fabien.maussion at gmail.com (Fabien) Date: Sun, 22 Feb 2015 12:59:49 +0100 Subject: pypandoc and restructured text References: Message-ID: On 22.02.2015 00:20, alb wrote: > I finally upgraded! And I'm currently trying out xfce! > Thanks again for the suggestions. > > Al > > p.s.: now pandoc works as expected. I don't want to sound insistent, but as a Linux user I personnaly recommend not to use "apt" to install and use python packages. Installed packages will soon or late become outdated or they will become a burden because you have less control on what you have installed and why. I really like virtualenv for it's help in keeping things (system / python version / fooling around with new packages) separated. It's also the recommendation of the recommendation of the "Python Packaging Authority": https://packaging.python.org/en/latest/current.html#installation-tool-recommendations Fabien From cfkaran2 at gmail.com Sun Feb 22 07:07:17 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 07:07:17 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> Message-ID: <1444051E-DBD0-4162-94DF-50BC6CF62A92@gmail.com> On Feb 21, 2015, at 10:55 AM, Chris Angelico wrote: > On Sun, Feb 22, 2015 at 2:45 AM, Cem Karan wrote: >> OK, so if I'm reading your code correctly, you're breaking the cycle in your object graph by making the GUI the owner of the callback, correct? No other chunk of code has a reference to the callback, correct? > > Correct. The GUI engine ultimately owns everything. Of course, this is > a very simple case (imagine a little notification popup; you don't > care about it, you don't need to know when it's been closed, the only > event on it is "hit Close to destroy the window"), and most usage > would have other complications, but it's not uncommon for me to build > a GUI program that leaves everything owned by the GUI engine. > Everything is done through callbacks. Destroy a window, clean up its > callbacks. The main window will have an "on-deletion" callback that > terminates the program, perhaps. It's pretty straight-forward. How do you handle returning information? E.g., the user types in a number and expects that to update the internal state of your code somewhere. Thanks, Cem Karan From cfkaran2 at gmail.com Sun Feb 22 07:10:31 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 07:10:31 -0500 Subject: Design thought for callbacks In-Reply-To: <87zj87i74p.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87zj87i74p.fsf@elektro.pacujo.net> Message-ID: On Feb 21, 2015, at 11:03 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: > >>> In order to inform users that certain bits of state have changed, I >>> require them to register a callback with my code. The problem is that >>> when I store these callbacks, it naturally creates a strong reference >>> to the objects, which means that if they are deleted without >>> unregistering themselves first, my code will keep the callbacks >>> alive. Since this could lead to really weird and nasty situations, >>> [...] >> >> No, it's not. I would advise using strong references - if the callback >> is a closure, for instance, you need to hang onto it, because there >> are unlikely to be any other references to it. If I register a >> callback with you, I expect it to be called; I expect, in fact, that >> that *will* keep my object alive. > > I use callbacks all the time but haven't had any problems with strong > references. > > I am careful to move my objects to a zombie state after they're done so > they can absorb any potential loose callbacks that are lingering in the > system. So, if I were designing a library for you, you would be willing to have a 'zombie' attribute on your callback, correct? This would allow the library to query its callbacks to ensure that only 'live' callbacks are called. How would you handle closures? Thanks, Cem Karan From marko at pacujo.net Sun Feb 22 07:12:52 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 14:12:52 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87zj87i74p.fsf@elektro.pacujo.net> Message-ID: <87k2za9mbf.fsf@elektro.pacujo.net> Cem Karan : > On Feb 21, 2015, at 11:03 AM, Marko Rauhamaa wrote: >> I use callbacks all the time but haven't had any problems with strong >> references. >> >> I am careful to move my objects to a zombie state after they're done so >> they can absorb any potential loose callbacks that are lingering in the >> system. > > So, if I were designing a library for you, you would be willing to have > a 'zombie' attribute on your callback, correct? This would allow the > library to query its callbacks to ensure that only 'live' callbacks are > called. How would you handle closures? Sorry, don't understand the question. Marko From cfkaran2 at gmail.com Sun Feb 22 07:16:14 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 07:16:14 -0500 Subject: Design thought for callbacks In-Reply-To: <87egpjtcp0.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> <87egpjtcp0.fsf@elektro.pacujo.net> Message-ID: <5204A93E-8834-4C6E-9B6C-F885A4B3D89A@gmail.com> On Feb 21, 2015, at 12:08 PM, Marko Rauhamaa wrote: > Steven D'Aprano : > >> Other than that, I cannot see how calling a function which has *not* >> yet been garbage collected can fail, just because the only reference >> still existing is a weak reference. > > Maybe the logic of the receiving object isn't prepared for the callback > anymore after an intervening event. > > The problem then, of course, is in the logic and not in the callbacks. This was PRECISELY the situation I was thinking about. My hope was to make the callback mechanism slightly less surprising by allowing the user to track them, releasing them when they aren't needed without having to figure out where the callbacks were registered. However, it appears I'm making things more surprising rather than less. Thanks, Cem Karan From rosuav at gmail.com Sun Feb 22 07:24:09 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Feb 2015 23:24:09 +1100 Subject: Design thought for callbacks In-Reply-To: <1444051E-DBD0-4162-94DF-50BC6CF62A92@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <1444051E-DBD0-4162-94DF-50BC6CF62A92@gmail.com> Message-ID: On Sun, Feb 22, 2015 at 11:07 PM, Cem Karan wrote: >> Correct. The GUI engine ultimately owns everything. Of course, this is >> a very simple case (imagine a little notification popup; you don't >> care about it, you don't need to know when it's been closed, the only >> event on it is "hit Close to destroy the window"), and most usage >> would have other complications, but it's not uncommon for me to build >> a GUI program that leaves everything owned by the GUI engine. >> Everything is done through callbacks. Destroy a window, clean up its >> callbacks. The main window will have an "on-deletion" callback that >> terminates the program, perhaps. It's pretty straight-forward. > > How do you handle returning information? E.g., the user types in a number and expects that to update the internal state of your code somewhere. Not sure what you mean by "returning". If the user types in a number in a GUI widget, that would trigger some kind of on-change event, and either the new text would be a parameter to the callback function, or the callback could query the widget. In the latter case, I'd probably have the callback as a closure, and thus able to reference the object. ChrisA From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Sun Feb 22 07:45:03 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Sun, 22 Feb 2015 12:45:03 +0000 Subject: Future of Pypy? Message-ID: As an engineer, I can quickly knock together behavioural models of electronic circuits, complete units, and control systems in Python, then annoyingly in a few recent cases, have to re-write in C for speed. I've tried PyPy, the just-in-time compiler for Python, and that is impressively, hugely fast in comparison, but it's no good making these models if I can't display the results in a useful way, and at the moment PyPy just doesn't have the huge range of useful time-saving libraries that CPython has. It's still quicker to do a re-write in the more cumbersome C than try to work with PyPy because C, like CPython, also has many useful libraries. A few years back, I recall people saying that PyPy was going to be the future of Python, but it seems to me that CPython still has the lion's share of the momentum, is developing faster and has ever more libraries, while PyPy is struggling to get enough workers to even get Numpy completed. Maybe there's not enough people like me that have really felt the need for the speed. Or maybe it's simply the accident of the historical development path that's set-in-stone an interpreter rather than a JIT. Anybody got a useful perspective on this? From marko at pacujo.net Sun Feb 22 07:46:15 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 14:46:15 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> <87egpjtcp0.fsf@elektro.pacujo.net> Message-ID: <87fv9y9krs.fsf@elektro.pacujo.net> Cem Karan : > On Feb 21, 2015, at 12:08 PM, Marko Rauhamaa wrote: >> Maybe the logic of the receiving object isn't prepared for the callback >> anymore after an intervening event. >> >> The problem then, of course, is in the logic and not in the callbacks. > > This was PRECISELY the situation I was thinking about. My hope was to > make the callback mechanism slightly less surprising by allowing the > user to track them, releasing them when they aren't needed without > having to figure out where the callbacks were registered. However, it > appears I'm making things more surprising rather than less. When dealing with callbacks, my advice is to create your objects as explicit finite state machines. Don't try to encode the object state implicitly or indirectly. Rather, give each and every state a symbolic name and log the state transitions for troubleshooting. Your callbacks should then consider what to do in each state. There are different ways to express this in Python, but it always boils down to a state/transition matrix. Callbacks sometimes cannot be canceled after they have been committed to and have been shipped to the event pipeline. Then, the receiving object must brace itself for the impending spurious callback. Marko From lac at openend.se Sun Feb 22 07:52:01 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 13:52:01 +0100 Subject: Design thought for callbacks In-Reply-To: Message from Cem Karan of "Sun, 22 Feb 2015 07:16:14 -0500." <5204A93E-8834-4C6E-9B6C-F885A4B3D89A@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> <87egpjtcp0.fsf@elektro.pacujo.net><5204A93E-8834-4C6E-9B6C-F885A4B3D89A@gmail.com> Message-ID: <201502221252.t1MCq11A012613@fido.openend.se> In a message of Sun, 22 Feb 2015 07:16:14 -0500, Cem Karan writes: >This was PRECISELY the situation I was thinking about. My hope was >to make the callback mechanism slightly less surprising by allowing >the user to track them, releasing them when they aren't needed >without having to figure out where the callbacks were registered. >However, it appears I'm making things more surprising rather than >less. You may be able to accomplish your goal by using a Queue with a producer/consumer model. see: http://stackoverflow.com/questions/9968592/turn-functions-with-a-callback-into-python-generators especially the bottom of that. I haven't run the code, but it looks mostly reasonable, except that you do not want to rely on the Queue maxsize being 1 here, and indeed, I almost always want a bigger Queue in any case. Use Queue.task_done if blocking the producer features in your design. The problem that you are up against is that callbacks are inherantly confusing, even to programmers who are learning about them for the first time. They don't fit people's internal model of 'how code works'. There isn't a whole lot one can do about that except to try to make the magic do as little as possible, so that more of the code works 'the way people expect'. Laura From jkn_gg at nicorp.f9.co.uk Sun Feb 22 07:58:00 2015 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Sun, 22 Feb 2015 04:58:00 -0800 (PST) Subject: Future of Pypy? In-Reply-To: References: Message-ID: On Sunday, 22 February 2015 12:45:15 UTC, Dave Farrance wrote: > As an engineer, I can quickly knock together behavioural models of > electronic circuits, complete units, and control systems in Python, then > annoyingly in a few recent cases, have to re-write in C for speed. > > I've tried PyPy, the just-in-time compiler for Python, and that is > impressively, hugely fast in comparison, but it's no good making these > models if I can't display the results in a useful way, and at the moment > PyPy just doesn't have the huge range of useful time-saving libraries that > CPython has. It's still quicker to do a re-write in the more cumbersome C > than try to work with PyPy because C, like CPython, also has many useful > libraries. > > A few years back, I recall people saying that PyPy was going to be the > future of Python, but it seems to me that CPython still has the lion's > share of the momentum, is developing faster and has ever more libraries, > while PyPy is struggling to get enough workers to even get Numpy > completed. > > Maybe there's not enough people like me that have really felt the need for > the speed. Or maybe it's simply the accident of the historical > development path that's set-in-stone an interpreter rather than a JIT. > Anybody got a useful perspective on this? I'm curious what ...behavioural... models you are creating quickly in Python that then need rewriting in C for speed. SPICE? some other CAD? Might be interesting to learn more about what and how you are actually doing. How about running your front end (simulation) work in PyPy, and the backend display work on CPython, if there are some missing features in PyPy that you need. This may be more or less easy depending on your requirements and any intermediate format you have. Or you could offer to assist in the PyPy porting? Or express an interest in specific libraries being ported? Cheers Jon N From skip.montanaro at gmail.com Sun Feb 22 08:04:31 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 22 Feb 2015 07:04:31 -0600 Subject: Standard In-Reply-To: References: Message-ID: On Thu, Feb 19, 2015 at 10:27 AM, Phillip Fleming wrote: > In my opinion, Python will not take off like C/C++ if there is no ANSI > standard. On one side of your statement, what makes you think Python ever wanted to "take off like C/C++"? On the other side, there are other languages (Java, PHP, Perl, Tcl) which have done pretty well without ANSI standardization. Python as well, as done fine in my opinion without an ANSI standard. I can't help but think I've just given a troll a carrot though... Skip From cfkaran2 at gmail.com Sun Feb 22 08:13:09 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 08:13:09 -0500 Subject: Design thought for callbacks In-Reply-To: <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> Message-ID: <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> On Feb 21, 2015, at 12:27 PM, Steven D'Aprano wrote: > Cem Karan wrote: > >> >> On Feb 21, 2015, at 8:15 AM, Chris Angelico wrote: >> >>> On Sun, Feb 22, 2015 at 12:13 AM, Cem Karan wrote: >>>> OK, so it would violate the principle of least surprise for you. >>>> Interesting. Is this a general pattern in python? That is, callbacks >>>> are owned by what they are registered with? >>>> >>>> In the end, I want to make a library that offers as few surprises to the >>>> user as possible, and no matter how I think about callbacks, they are >>>> surprising to me. If callbacks are strongly-held, then calling 'del >>>> foo' on a callable object may not make it go away, which can lead to >>>> weird and nasty situations. > > How? > > The whole point of callbacks is that you hand over responsibility to another > piece of code, and then forget about your callback. The library will call > it, when and if necessary, and when the library no longer needs your > callback, it is free to throw it away. (If I wish the callback to survive > beyond the lifetime of your library's use of it, I have to keep a reference > to the function.) Marko mentioned it earlier; if you think you've gotten rid of all references to some chunk of code, and it is still alive afterwards, that can be surprising. >>>> Weakly-held callbacks mean that I (as the >>>> programmer), know that objects will go away after the next garbage >>>> collection (see Frank's earlier message), so I don't get 'dead' >>>> callbacks coming back from the grave to haunt me. > > I'm afraid this makes no sense to me. Can you explain, or better still > demonstrate, a scenario where "dead callbacks rise from the grave", so to > speak? """ #! /usr/bin/env python class Callback_object(object): def __init__(self, msg): self._msg = msg def callback(self, stuff): print("From {0!s}: {1!s}".format(self._msg, stuff)) class Fake_library(object): def __init__(self): self._callbacks = list() def register_callback(self, callback): self._callbacks.append(callback) def execute_callbacks(self): for thing in self._callbacks: thing('Surprise!') if __name__ == "__main__": foo = Callback_object("Evil Zombie") lib = Fake_library() lib.register_callback(foo.callback) # Way later, after the user forgot all about the callback above foo = Callback_object("Your Significant Other") lib.register_callback(foo.callback) # And finally getting around to running all those callbacks. lib.execute_callbacks() """ Output: From Evil Zombie: Surprise! From Your Significant Other: Surprise! In this case, the user made an error (just as Marko said in his earlier message), and forgot about the callback he registered with the library. The callback isn't really rising from the dead; as you say, either its been garbage collected, or it hasn't been. However, you may not be ready for a callback to be called at that moment in time, which means you're surprised by unexpected behavior. >>>> So, what's the consensus on the list, strongly-held callbacks, or >>>> weakly-held ones? >>> >>> I don't know about Python specifically, but it's certainly a general >>> pattern in other languages. They most definitely are owned, and it's >>> the only model that makes sense when you use closures (which won't >>> have any other references anywhere). >> >> I agree about closures; its the only way they could work. > > *scratches head* There's nothing special about closures. You can assign them > to a name like any other object. > > def make_closure(): > x = 23 > def closure(): > return x + 1 > return closure > > func = make_closure() > > Now you can register func as a callback, and de-register it when your done: > > register(func) > unregister(func) > > > Of course, if you thrown away your reference to func, you have no (easy) way > of de-registering it. That's no different to any other object which is > registered by identity. (Registering functions by name is a bad idea, since > multiple functions can have the same name.) > > As an alternative, your callback registration function might return a ticket > for the function: > > ticket = register(func) > del func > unregister(ticket) > > but that strikes me as over-kill. And of course, the simplest ticket is to > return the function itself :-) Agreed on all points; closures are just ordinary objects. The only difference (in my opinion) is that they are 'fire and forget'; if you are registering or tracking them then you've kind of defeated the purpose. THAT is what I meant about how you handle closures. > >> When I was >> originally thinking about the library, I was trying to include all types >> of callbacks, including closures and callable objects. The callable >> objects may pass themselves, or one of their methods to the library, or >> may do something really weird. > > I don't think they can do anything too weird. They have to pass a callable > object. Your library just calls that object. You shouldn't need to care > whether it is a function, a method, a type, a callable instance, or > something else. You just call it, and when you're done calling it forever, > you just throw it away. That doesn't quite solve the problem, but it comes close. The headache (as shown in my earlier code) is that you think you've gotten rid of something before it is called, but it turns out you haven't. I'm starting to think that there isn't a solution to this other than telling the programmer "Don't do that". >> Although I just realized that closures may cause another problem. In my >> code, I expect that many different callbacks can be registered for the >> same event. Unregistering means you request to be unregistered for the >> event. How do you do that with a closure? Aren't they anonymous? > > Not unless you create them using lambda. Using the make_closure function > above: > > > py> func = make_closure() > py> func.__name__ > 'closure' > > Of course, if you call make_closure twice, both functions will have the same > internal name. You can set the function __name__ and __qualname__ to fix > that. This is how the functools.wraps decorator works. > > But that's a red herring. Don't register functions by name! Not all callable > objects have names, and those that do, you may have multiple *distinct* > callbacks with the same name. > > There are two reasonable approaches: unregister by identity, or by returning > a ticket which uniquely identifies the callback. The user is responsible > for keeping track of their own ticket. If I lose it, I can't unregister my > callback any more. So sad, sucks to be me. > > > The simplest possible identity-based scheme would be something like this: > > > # don't hate me for using a global variable > CALLBACKS = [] > > def register(func): > if func not in CALLBACKS: > CALLBACKS.append(func) > > def unregister(func): > try: > CALLBACKS.remove(func) > except ValueError: > pass > > > That's probably a bit too simple, since it won't behave as expected with > bound methods. The problem is that bound methods are generated on the fly, > so this won't work: > > register(instance.spam) > # later > unregister(instance.spam) # a different instance! > > I would have to do this: > > bound_method = instance.spam > register(bound_method) > unregister(bound_method) > > > But a more sophisticated unregister function should work: > > # Untested > def unregister(func): > for i, f in enumerate(CALLBACKS): > if (f is func) or (isinstance(f, types.MethodType) > and f.__wrapped__ is func): > del CALLBACKS[i] > return Are you sure about that? I just tested out the following code, and it appears to work correctly: """ #! /usr/bin/env python class Callback_object(object): def __init__(self, msg): self._msg = msg def callback(self, stuff): print("From {0!s}: {1!s}".format(self._msg, stuff)) class Fake_library(object): def __init__(self): self._callbacks = set() def register_callback(self, callback): self._callbacks.add(callback) def unregister_callback(self, callback): self._callbacks.discard(callback) def execute_callbacks(self): for thing in self._callbacks: thing('Surprise!') if __name__ == "__main__": foo = Callback_object("Evil Zombie") lib = Fake_library() lib.register_callback(foo.callback) lib.unregister_callback(foo.callback) lib.execute_callbacks() """ I'll admit though, I don't know if it worked because I got lucky, or if python guarantees it works... > The simplest possible ticket-based system is probably something like this: > > CALLBACKS = {} > NEXT_TICKET = 1 > > def register(func): > global NEXT_TICKET > ticket = NEXT_TICKET > NEXT_TICKET += 1 > callbacks[ticket] = func > return ticket > > def unregister(ticket): > if ticket in CALLBACKS: > del CALLBACKS[ticket] > I'd probably go with something similar to this, except that I'd use UUIDs for the tickets. I know me and my users, and somewhere along the line I'd use a ticket to unregister from the wrong callback dictionary! >>> If you're expecting 'del foo' to destroy the object, then you have a >>> bigger problem than callbacks, because that's simply not how Python >>> works. You can't _ever_ assume that deleting something from your local >>> namespace will destroy the object, because there can always be more >>> references. So maybe you need a more clear way of saying "I'm done >>> with this, get rid of it". >> >> Agreed about 'del', and I don't assume that the object goes away at the >> point. The problem is debugging and determining WHY your object is still >> around. I know a combination of logging and gc.get_referrers() will >> probably help you figure out why something is still around, but I'm trying >> to avoid that headache. > > Why do you care? Surely all your library should care about is whether or not > they have a reference to the callback.If they do, they should call it (when > appropriate). If they don't, they aren't responsible for it. I care because I care about anyone using my code. Telling them 'tough, its your problem' doesn't get you many friends. Making a library that performs as expected and where it is easy to debug what went wrong makes everyone (including me!) happy. >> I guess the real problem is how this creates cycles in the call graph. >> User code effectively owns the library code, which via callbacks owns the >> user code. I have no idea what the best point the cycle is to break it, >> and not surprise someone down the road. The only idea I have is to >> redesign the library a little, and make anything that accepts a callback >> actually be a subclass of collections.abc.Container, or even >> collections.abc.MutableSet. That makes it very obvious that the object >> owns the callback, and that you will need to remove your object to >> unregister it. > > My brain hurts from the complexity of your solution. What is the actual > problem you are trying to solve? I would like to see an example of an > actual failure before trying to solve a purely theoretical failure mode. If I'm making your head hurt, then my solution is a bad solution. The whole reason I started this discussion was to figure out if an alternative method would make more sense to my potential endusers (fellow programmers). If strongly-held callbacks cause fewer headaches, then that is what I'll go with. Thanks, Cem Karan From cfkaran2 at gmail.com Sun Feb 22 08:21:35 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 08:21:35 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <762D842A-A05A-47DD-9C8E-B1E0BB616DED@gmail.com> On Feb 21, 2015, at 3:57 PM, Grant Edwards wrote: > On 2015-02-21, Cem Karan wrote: >> >> On Feb 21, 2015, at 12:42 AM, Chris Angelico wrote: >> >>> On Sat, Feb 21, 2015 at 1:44 PM, Cem Karan wrote: >>>> In order to inform users that certain bits of state have changed, I require them to register a callback with my code. The problem is that when I store these callbacks, it naturally creates a strong reference to the objects, which means that if they are deleted without unregistering themselves first, my code will keep the callbacks alive. Since this could lead to really weird and nasty situations, I would like to store all the callbacks in a WeakSet (https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That way, my code isn't the reason why the objects are kept alive, and if they are no longer alive, they are automatically removed from the WeakSet, preventing me from accidentally calling them when they are dead. My question is simple; is this a good design? If not, why not? Are there any potential 'gotchas' I should be worried about? >>>> >>> >>> No, it's not. I would advise using strong references - if the callback >>> is a closure, for instance, you need to hang onto it, because there >>> are unlikely to be any other references to it. If I register a >>> callback with you, I expect it to be called; I expect, in fact, that >>> that *will* keep my object alive. >> >> OK, so it would violate the principle of least surprise for you. > > And me as well. I would expect to be able to pass a closure as a > callback and not have to keep a reference to it. Perhaps that just a > leftover from working with other languages (javascript, scheme, etc.). > It doesn't matter if it's a string, a float, a callback, a graphic or > whatever: if I pass your function/library an object, I expect _you_ to > keep track of it until you're done with it. > >> Interesting. Is this a general pattern in python? That is, >> callbacks are owned by what they are registered with? > > I'm not sure what you mean by "owned" or why it matters that it's a > callback: it's an object that was passed to you: you need to hold onto > a reference to it until you're done with it, and the polite thing to > do is to delete references to it when you're done with it. I tend to structure my code as a tree or DAG of objects. The owner refers to the owned object, but the owned object has no reference to its owner. With callbacks, you get cycles, where the owned owns the owner. As a result, if you forget where your object has been registered, it may be kept alive when you aren't expecting it. My hope was that with WeakSets I could continue to preserve the DAG or tree while still having the benefits of callbacks. However, it looks like that is too surprising to most people. Thanks, Cem Karan From lac at openend.se Sun Feb 22 08:27:46 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 14:27:46 +0100 Subject: Future of Pypy? In-Reply-To: Message from Dave Farrance of "Sun, 22 Feb 2015 12:45:03 +0000." References: Message-ID: <201502221327.t1MDRkIH019657@fido.openend.se> In a message of Sun, 22 Feb 2015 12:45:03 +0000, Dave Farrance writes: >Maybe there's not enough people like me that have really felt the need for >the speed. Or maybe it's simply the accident of the historical >development path that's set-in-stone an interpreter rather than a JIT. >Anybody got a useful perspective on this? I don't understand 'an interpreter rather than a JIT'. PyPy has a JIT, that sort of is the whole point. One problem is that hacking on PyPy itself is hard. Lots of people find it too hard, and give up. (Of course, lots of people give up on hacking CPython too. I think that hacking on PyPy is harder than hacking on CPython, but I am quite biased.) So this is a barrier to getting more people to work on it. Provided your code is in pure python, PyPy already works great for you. as you found out. Pure Python libraries aren't a problem, either. The problem arises when you want to add your fancy graphic library to what you do, because chances are that fancy library is a C extension. And there is no magic 'sprinke this dust here' on a C extension that makes it acceptable to PyPy. It's a hard job. The PyPy team has gone after, and rewritten how to do this I think 5 times now. Maybe more. Every time the goal has been to make it easier for the average programmer to make an interface, and, of course to not slow things down too much. C extensions, in general, make PyPy code run a lot slower because you cannot get the JIT in there to speed things up, so you may be stuck with unjitted PyPy performance, even on your python code, which isn't speedy. You also find lots of bugs in the C extensions, which don't get noticed until you, for instance, no longer have a ref counting GC. Some of the things aren't really bugs, exactly, just that the person who wrote the thing knew far, far, too much about how CPython works and has produced something that had no need or desire to be portable anywhere else. The closer the person who wrote the extension was 'to the metal' .. knowing _exactly_ how CPython does things and how to squeeze that for the tiniest last drop of performance improvement .... the harder things are for whoever wants to get it to work with PyPy, which has a competely different architecture and a whole lot of other assumptions. And the slower that it will run. So, if it is a small thing, then the usual suggestion is to rewrite it in pure python and let the JIT handle it. Very, very often the result is faster than the pure C code, but clearly this isn't something you want to do with a huge graphics library ... There is hope that we can take another crack at this problem using things learned from the Transactional Memory stuff, but nobody is promising anything yet. Also Armin has had a new neat thought. https://mail.python.org/pipermail/pypy-dev/2015-February/013085.html If we can get rid of the warmup time, then PyPy should be more popular than it is now. Lots of people run PyPy once, un-warmed up, see no big improvement, and decide it's not for them. But the problem of 'here is a huge chunk of C code, designed to work with Language X. Now make it work with Language Y (PyPy) which isn't written in C' can only be simplified to a certain extent. There comes a point where _this is just bloody hard_. Laura From cfkaran2 at gmail.com Sun Feb 22 08:42:10 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 08:42:10 -0500 Subject: Design thought for callbacks In-Reply-To: <87k2za9mbf.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87zj87i74p.fsf@elektro.pacujo.net> <87k2za9mbf.fsf@elektro.pacujo.net> Message-ID: <9E0EB50F-F794-42FF-8F4C-A426ED0FF6C1@gmail.com> On Feb 22, 2015, at 7:12 AM, Marko Rauhamaa wrote: > Cem Karan : > >> On Feb 21, 2015, at 11:03 AM, Marko Rauhamaa wrote: >>> I use callbacks all the time but haven't had any problems with strong >>> references. >>> >>> I am careful to move my objects to a zombie state after they're done so >>> they can absorb any potential loose callbacks that are lingering in the >>> system. >> >> So, if I were designing a library for you, you would be willing to have >> a 'zombie' attribute on your callback, correct? This would allow the >> library to query its callbacks to ensure that only 'live' callbacks are >> called. How would you handle closures? > > Sorry, don't understand the question. You were saying that you move your objects into a zombie state. I assumed that you meant you marked them in some manner (e.g., setting 'is_zombie' to True), so that anything that has a strong reference to the object knows the object is not supposed to be used anymore. That way, regardless of where or how many times you've registered your object for callbacks, the library can do something like the following (banged out in my mail application, may have typos): """ _CALLBACKS = [] def execute_callbacks(): global _CALLBACKS _CALLBACKS = [x for x in _CALLBACKS if not x.is_zombie] for x in _CALLBACKS: x() """ That will lazily unregister callbacks that are in the zombie state, which will eventually lead to their collection by the garbage collector. It won't work for anything that you don't have a reference for (lambdas, etc.), but it should work in a lot of cases. Is this what you meant? Thanks, Cem Karan From steve+comp.lang.python at pearwood.info Sun Feb 22 08:45:49 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 00:45:49 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> <54e9b027$0$12992$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54e9dd8e$0$12975$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Sun, Feb 22, 2015 at 9:32 PM, Steven D'Aprano > wrote: >> Why? Do you expect that the Python garbage collector special cases >> callbacks to keep them alive even when there are no references to them? >> How would it distinguish a callback from some other function? > > No no no. It's the other way around. _Something_ has to be doing those > callbacks, and it's that _something_ that should be keeping them > alive. The fact that it's a registered callback should itself *be* a > reference (and not a weak reference), and should keep it alive. That's much more reasonable than what you said earlier: it seems wrong to have to stash a thing in a bucket in order to keep its callbacks alive. I expect the callbacks themselves to keep it alive. So yes. If I bind a callback to a button, say, or a listener, then the button (or listener) keeps the callback alive, *not* the callback keeping the button or listener alive. But if there are no references to the button, or the listener, then it will be garbage-collected, which will free the references to the callback and allow it to be garbage-collected as well (if there are no further references to it). -- Steven From marko at pacujo.net Sun Feb 22 08:54:04 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 15:54:04 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87zj87i74p.fsf@elektro.pacujo.net> <87k2za9mbf.fsf@elektro.pacujo.net> Message-ID: <8761au9hmr.fsf@elektro.pacujo.net> Cem Karan : > You were saying that you move your objects into a zombie state. I > assumed that you meant you marked them in some manner (e.g., setting > 'is_zombie' to True), Yes, but even better: self.set_state(ZOMBIE) > so that anything that has a strong reference to the object knows the > object is not supposed to be used anymore. The other way round: the zombie object knows to ignore callbacks sent its way. It's not the responsibility of the sender to mind the receiver's internal state. I nowadays tend to implement states as inner classes. Here's how I've implemented the zombie state of one class: class Delivery...: def __init__(...): ... class ZOMBIE(STATE): def handle_connected(self): pass def handle_eof(self): pass def handle_response(self, code, response): pass def handle_io_error(self, errcode): pass def zombifie(self): assert False def transaction_timeout(self): assert False Marko From cfkaran2 at gmail.com Sun Feb 22 08:54:28 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 08:54:28 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <1444051E-DBD0-4162-94DF-50BC6CF62A92@gmail.com> Message-ID: <44E35377-0E1A-4C41-82E7-FB5BC1C158CB@gmail.com> On Feb 22, 2015, at 7:24 AM, Chris Angelico wrote: > On Sun, Feb 22, 2015 at 11:07 PM, Cem Karan wrote: >>> Correct. The GUI engine ultimately owns everything. Of course, this is >>> a very simple case (imagine a little notification popup; you don't >>> care about it, you don't need to know when it's been closed, the only >>> event on it is "hit Close to destroy the window"), and most usage >>> would have other complications, but it's not uncommon for me to build >>> a GUI program that leaves everything owned by the GUI engine. >>> Everything is done through callbacks. Destroy a window, clean up its >>> callbacks. The main window will have an "on-deletion" callback that >>> terminates the program, perhaps. It's pretty straight-forward. >> >> How do you handle returning information? E.g., the user types in a number and expects that to update the internal state of your code somewhere. > > Not sure what you mean by "returning". If the user types in a number > in a GUI widget, that would trigger some kind of on-change event, and > either the new text would be a parameter to the callback function, or > the callback could query the widget. In the latter case, I'd probably > have the callback as a closure, and thus able to reference the object. We're thinking of the same thing. I try to structure what little GUI code I write using the MVP pattern (http://en.wikipedia.org/wiki/Model-view-presenter), so I have these hub and spoke patterns. But you're right, if you have a partially evaluated callback that has the presenter as one of the parameters, that would do it for a GUI. I was thinking more of a DAG of objects, but now that I think about it, callbacks wouldn't make sense in that case. Thanks, Cem Karan From steve+comp.lang.python at pearwood.info Sun Feb 22 08:57:24 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 00:57:24 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> <87ioeui9zd.fsf@elektro.pacujo.net> Message-ID: <54e9e045$0$12988$c3e8da3$5496439d@news.astraweb.com> Marko Rauhamaa wrote: > Chris Angelico : > >> On Sun, Feb 22, 2015 at 7:34 PM, Marko Rauhamaa wrote: >>> Refloops are not to be worried about, let alone removed. >> >> Why? > > Because the whole point of GC-languages is that you should stop worrying > about memory. Trying to mastermind and micromanage GC in the application > is, pardon my French, an antipattern. While it would be nice to be able to stop worrying about memory, try to calculate 1000**1000**1000 and see how that works for you. Garbage collection enables us to *mostly* automate the allocation and deallocation of memory. If doesn't mean we can forget about it. GC is an abstraction that frees us from most of the grunt work of allocating memory, but it doesn't mean that there is never any need to think about memory. GC is a leaky abstraction. Depending on the implementation, it may cause distracting and annoying pauses in your application and/or resource leaks. Even if there are no pauses, GC still carries a performance penalty. Good programmers need to be aware of the limitations of their tools, and be prepared to code accordingly. When writing programs for educational purposes, we should try to code in the simplest and most elegant way with no thought given to annoying practical matters. At least at first. But when writing programs for actual use, we should write for the implication we have, not the one we wish we had. >> They force the use of the much slower cycle-detecting GC, rather than >> the quick and efficient CPython refcounter. > > Java's Hotspot doesn't bother with refcounters but is much faster than > Python. CPython's refcounters are a historical accident that a Python > application developer shouldn't even be aware of. I don't know about Java's Hotspot, but I do know that CPython's ref counting garbage collector has at least one advantage over the GC used by Jython and IronPython: unlike them, open files are closed as soon as they are no longer in use. Code like this may run out of operating system file handles in Jython: i = 0 while True: f = open('/tmp/x%d' % i) i += 1 while CPython will just keep going. I suppose it will *eventually* run out of some resource, but probably not file handles. Oh, a bit of trivia: Apple is abandoning their garbage collector and going back to a reference counter: https://developer.apple.com/news/?id=02202015a Word on Reddit is that Apple is concerned about performance and battery life. P.S. A reminder that reference counting *is* a form of garbage collection. >> I don't know how other Pythons work, but mark-and-sweep has its own >> costs, and I don't know of any system that's both prompt and able to >> detect refloops. > > It's exceedingly difficult (and pointless) to detect cycles in your > object structures. Python is going to have to do a GC occasionally > anyway. Yes, your worst-case response times are going to suffer, but > that's the cost of doing business. In *general*, you're right. Who wants to spend all their time worrying about cycles when the GC can do it for you? But if cycles are rare, and in known parts of your code where it is simple to break them when you're done, there's no disadvantage to doing so. Leave the GC for the hard cases. It's like explicitly closing a file, either with file.close() or a context manager. When using CPython, it doesn't really matter whether you close the file or not, since the ref counter will normally close it automatically as soon as the file goes out of scope. But it is cheap and easy to do so, so why not do it? Then, when it otherwise would matter, say you are running under Jython, it doesn't because you've closed the file. >> Helping it along means your program doesn't waste memory. Why such a >> blanket statement? > > Because worrying Python programmers with evil spirits (reference loops) > leads to awkward coding practices and takes away one of the main > advantages of Python as a high-level programming language. I think you exaggerate a tad. We're not trying to scare beginners, we're a group of moderately experienced coders discussing "best practice" (or at least "reasonable practice") when using callbacks. -- Steven From rosuav at gmail.com Sun Feb 22 09:01:06 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 01:01:06 +1100 Subject: Design thought for callbacks In-Reply-To: <54e9dd8e$0$12975$c3e8da3$5496439d@news.astraweb.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <54e95d34$0$13006$c3e8da3$5496439d@news.astraweb.com> <54e9b027$0$12992$c3e8da3$5496439d@news.astraweb.com> <54e9dd8e$0$12975$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 23, 2015 at 12:45 AM, Steven D'Aprano wrote: >> No no no. It's the other way around. _Something_ has to be doing those >> callbacks, and it's that _something_ that should be keeping them >> alive. The fact that it's a registered callback should itself *be* a >> reference (and not a weak reference), and should keep it alive. > > That's much more reasonable than what you said earlier: > > it seems wrong to have to stash a thing in a bucket in order > to keep its callbacks alive. I expect the callbacks themselves to > keep it alive. > > > So yes. If I bind a callback to a button, say, or a listener, then the > button (or listener) keeps the callback alive, *not* the callback keeping > the button or listener alive. I meant the same thing, but my terminology was poor. Yes, that's correct; it's not any sort of magic about it being a callback, but more that the one you register it with becomes the owner of something. Hence, no weak references. ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 22 09:05:41 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 01:05:41 +1100 Subject: Future of Pypy? References: Message-ID: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> Dave Farrance wrote: > As an engineer, I can quickly knock together behavioural models of > electronic circuits, complete units, and control systems in Python, then > annoyingly in a few recent cases, have to re-write in C for speed. > > I've tried PyPy, the just-in-time compiler for Python, and that is > impressively, hugely fast in comparison, but it's no good making these > models if I can't display the results in a useful way, and at the moment > PyPy just doesn't have the huge range of useful time-saving libraries that > CPython has. I assume you're talking about drawing graphics rather than writing text. Can you tell us which specific library or libraries won't run under PyPy? -- Steven From marko at pacujo.net Sun Feb 22 09:06:53 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 16:06:53 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <87egpjapxv.fsf@elektro.pacujo.net> <54e9391b$0$13010$c3e8da3$5496439d@news.astraweb.com> <87r3tiids7.fsf@elektro.pacujo.net> <87mw46ibtq.fsf@elektro.pacujo.net> <87ioeui9zd.fsf@elektro.pacujo.net> <54e9e045$0$12988$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87y4nq82gy.fsf@elektro.pacujo.net> Steven D'Aprano : > I don't know about Java's Hotspot, but I do know that CPython's ref counting > garbage collector has at least one advantage over the GC used by Jython and > IronPython: unlike them, open files are closed as soon as they are no > longer in use. You can't depend on that kind of behavior. Dangling resources may or may not be cleaned up, ever. > Oh, a bit of trivia: Apple is abandoning their garbage collector and going > back to a reference counter: > > https://developer.apple.com/news/?id=02202015a > > Word on Reddit is that Apple is concerned about performance and battery > life. That truly is a bit OT here. > It's like explicitly closing a file, either with file.close() or a context > manager. Both methods are explicit. Closing files and other resources are not directly related to GC. Here's the thing: GC relieves your from dynamic memory management. You are still on your own when it comes to other resources. > We're not trying to scare beginners, we're a group of moderately > experienced coders discussing "best practice" (or at least "reasonable > practice") when using callbacks. Who mentioned beginners? I'm abiding by the same best practices I'm advocating. Marko From cfkaran2 at gmail.com Sun Feb 22 09:10:20 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 09:10:20 -0500 Subject: Design thought for callbacks In-Reply-To: <201502221252.t1MCq11A012613@fido.openend.se> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> <87egpjtcp0.fsf@elektro.pacujo.net><5204A93E-8834-4C6E-9B6C-F885A4B3D89A@gmail.com> <201502221252.t1MCq11A012613@fido.openend.se> Message-ID: <6CBD9C73-455D-4960-871E-F9B39146DF6E@gmail.com> On Feb 22, 2015, at 7:52 AM, Laura Creighton wrote: > In a message of Sun, 22 Feb 2015 07:16:14 -0500, Cem Karan writes: > >> This was PRECISELY the situation I was thinking about. My hope was >> to make the callback mechanism slightly less surprising by allowing >> the user to track them, releasing them when they aren't needed >> without having to figure out where the callbacks were registered. >> However, it appears I'm making things more surprising rather than >> less. > > You may be able to accomplish your goal by using a Queue with a > producer/consumer model. > see: http://stackoverflow.com/questions/9968592/turn-functions-with-a-callback-into-python-generators > > especially the bottom of that. > > I haven't run the code, but it looks mostly reasonable, except that > you do not want to rely on the Queue maxsize being 1 here, and > indeed, I almost always want a bigger Queue in any case. Use > Queue.task_done if blocking the producer features in your design. > > The problem that you are up against is that callbacks are inherantly > confusing, even to programmers who are learning about them for the > first time. They don't fit people's internal model of 'how code works'. > There isn't a whole lot one can do about that except to > try to make the magic do as little as possible, so that more of the > code works 'the way people expect'. I think what you're suggesting is that library users register a Queue instead of a callback, correct? The problem is that I'll then have a strong reference to the Queue, which means I'll be pumping events into it after the user code has gone away. I was hoping to solve the problem of forgotten registrations in the library. Thanks, Cem Karan From breamoreboy at yahoo.co.uk Sun Feb 22 09:11:16 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 22 Feb 2015 14:11:16 +0000 Subject: Standard In-Reply-To: References: Message-ID: On 19/02/2015 16:27, Phillip Fleming wrote: > In my opinion, Python will not take off like C/C++ if there is no ANSI > standard. > Python has already taken off because it doesn't have a standard as such. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python.list at tim.thechases.com Sun Feb 22 09:17:36 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 22 Feb 2015 08:17:36 -0600 Subject: What behavior would you expect? In-Reply-To: References: Message-ID: <20150222081736.0321d5f8@bigbox.christie.dr> On 2015-02-19 22:55, Jason Friedman wrote: > > If you're going to call listdir, you probably want to use fnmatch > > directly. > > > > fnmatch seems to be silent on non-existent directories: > python -c 'import fnmatch; fnmatch.fnmatch("/no/such/path", "*")' a better test would be glob.glob as fnmatch simply asks "does this string match this pattern?" so it cares nothing for filenames. However, it still holds that glob.glob("/does/not/exist/*.txt") doesn't raise an error but rather just returns an empty list of iterables. However, for the OP's question, it's max() that raises an error: import glob import os def most_recent_file(loc, pattern): globstr = os.path.join(loc, pattern) return max(glob.glob(globstr), key=lambda f: os.stat(f).st_mtime) gives me this when the glob returns an empty iterable: Traceback (most recent call last): File "", line 1, in ValueError: max() arg is an empty sequence -tkc From cfkaran2 at gmail.com Sun Feb 22 09:17:41 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 09:17:41 -0500 Subject: Design thought for callbacks In-Reply-To: <87fv9y9krs.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> <87egpjtcp0.fsf@elektro.pacujo.net> <87fv9y9krs.fsf@elektro.pacujo.net> Message-ID: <7CD1CDAB-6524-4D99-8561-59876D9C07F8@gmail.com> On Feb 22, 2015, at 7:46 AM, Marko Rauhamaa wrote: > Cem Karan : > >> On Feb 21, 2015, at 12:08 PM, Marko Rauhamaa wrote: >>> Maybe the logic of the receiving object isn't prepared for the callback >>> anymore after an intervening event. >>> >>> The problem then, of course, is in the logic and not in the callbacks. >> >> This was PRECISELY the situation I was thinking about. My hope was to >> make the callback mechanism slightly less surprising by allowing the >> user to track them, releasing them when they aren't needed without >> having to figure out where the callbacks were registered. However, it >> appears I'm making things more surprising rather than less. > > When dealing with callbacks, my advice is to create your objects as > explicit finite state machines. Don't try to encode the object state > implicitly or indirectly. Rather, give each and every state a symbolic > name and log the state transitions for troubleshooting. > > Your callbacks should then consider what to do in each state. There are > different ways to express this in Python, but it always boils down to a > state/transition matrix. > > Callbacks sometimes cannot be canceled after they have been committed to > and have been shipped to the event pipeline. Then, the receiving object > must brace itself for the impending spurious callback. Nononono, I'm NOT encoding anything implicitly! As Frank mentioned earlier, this is more of a pub/sub problem. E.g., 'USB dongle has gotten plugged in', or 'key has been pressed'. The user code needs to decide what to do next, the library code provides a nice, clean interface to some potentially weird hardware. Thanks, Cem Karan From cfkaran2 at gmail.com Sun Feb 22 09:22:45 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 09:22:45 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Feb 22, 2015, at 5:15 AM, Gregory Ewing wrote: > Frank Millman wrote: >> "In order to inform users that certain bits of state have changed, I require them to register a callback with my code." >> This sounds to me like a pub/sub scenario. When a 'listener' object comes into existence it is passed a reference to a 'controller' object that holds state. It wants to be informed when the state changes, so it registers a callback function with the controller. > > Perhaps instead of registering a callback function, you > should be registering the listener object together with > a method name. > > You can then keep a weak reference to the listener object, > since if it is no longer referenced elsewhere, it presumably > no longer needs to be notified of anything. I see what you're saying, but I don't think it gains us too much. If I store an object and an unbound method of the object, or if I store the bound method directly, I suspect it will yield approximately the same results. Thanks, Cem Karan From steve+comp.lang.python at pearwood.info Sun Feb 22 09:31:39 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 01:31:39 +1100 Subject: Standard References: Message-ID: <54e9e84b$0$12997$c3e8da3$5496439d@news.astraweb.com> Skip Montanaro wrote: > On Thu, Feb 19, 2015 at 10:27 AM, Phillip Fleming > wrote: >> In my opinion, Python will not take off like C/C++ if there is no ANSI >> standard. > > On one side of your statement, what makes you think Python ever wanted > to "take off like C/C++"? On the other side, there are other languages > (Java, PHP, Perl, Tcl) which have done pretty well without ANSI > standardization. Python as well, as done fine in my opinion without an > ANSI standard. I'm pretty sure that Python is doing pretty well, popularity-wise. Depending on how you measure it, it is even more popular than C! http://import-that.dreamwidth.org/1388.html I don't actually believe that Python is more popular than C. It's just that there is no one single definition of popularity for programming languages, so depending on how you do your measurement, you get different results. In any case, Python is consistently in the top dozen or so languages. Any suggestion that Python "will not take off" is laughably wrong and a decade too late. > I can't help but think I've just given a troll a carrot though... Very likely :-) -- Steven From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Sun Feb 22 10:30:36 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Sun, 22 Feb 2015 15:30:36 +0000 Subject: Future of Pypy? References: Message-ID: <5qpjea5125jcq3d18nvn91hb9mjun95dif@4ax.com> jkn wrote: > I'm curious what ...behavioural... models you are creating quickly in > Python that then need rewriting in C for speed. SPICE? some other CAD? > Might be interesting to learn more about what and how you are actually > doing. The convert-to-C cases were complex filtering functions. I do make good use of spice-based tools, but I often find it useful to make a more abstract model, usually before completing the design. This helps with component selection, finalizing the design, and making sure that I understood the what the circuit would do. I started work 1980ish, had an early 6502-based home computer, and my then place of work had some 6502-based Pet computers, so I gained the ability to quickly write BASIC programs as an engineering aid. Later, when BASIC dropped into obscurity, I switched to C and C++, although I always found that cumbersome compared to the old BASIC. Later still, when I found that my Google queries for code examples started returning more Python than C, I tried that -- and discovered that Python was like BASIC, only better. But that's just me. Other hardware engineers use a variety of modeling applications. Or don't need to because they're just that clever? Or they give the modeling work to system engineers who will use whatever apps that system engineers use, and will return a result a few weeks later. Personally, I've tended to get used to writing code in just one general-purpose language, and it seems to me that I get a useful result relatively quickly. > How about running your front end (simulation) work in PyPy, and the > backend display work on CPython, if there are some missing features in > PyPy that you need. This may be more or less easy depending on your > requirements and any intermediate format you have. Maybe I should look at that again. In the case of the filter models, their usefulness had grown to the point that requiring support by other people was a possibility, so converting them to C seemed better than writing something that bridged between two language implementations. > Or you could offer to assist in the PyPy porting? Or express an interest > in specific libraries being ported? I'm a hardware engineer not a software engineer, so I have to plead lack of ability there. I do appreciate the work that's done on Python, and have to be grateful for what is available, since I'm not paying for it. From joepureddu at yahoo.com Sun Feb 22 10:33:22 2015 From: joepureddu at yahoo.com (Dieffe Diciotto) Date: Sun, 22 Feb 2015 07:33:22 -0800 (PST) Subject: -- redacted -- Message-ID: -- redacted -- From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Sun Feb 22 10:36:42 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Sun, 22 Feb 2015 15:36:42 +0000 Subject: Future of Pypy? References: of "Sun, 22 Feb 2015 12:45:03 +0000." Message-ID: Laura Creighton wrote: >I don't understand 'an interpreter rather than a JIT'. PyPy has a >JIT, that sort of is the whole point. Yes. I meant that from my end-user, non-software-engineer perspective, it looked as though CPython was proceeding with leaps and bounds and that PyPy remained mostly a proof-of-concept after several years. But thanks for your description of the issues. So once the core issues are sorted out, if enough people can be found to work on it, then hopefully the library conversions will follow apace. From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Sun Feb 22 10:44:48 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Sun, 22 Feb 2015 15:44:48 +0000 Subject: Future of Pypy? References: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: >I assume you're talking about drawing graphics rather than writing text. Can >you tell us which specific library or libraries won't run under PyPy? Yes, mainly the graphics. I'm a hardware engineer, not a software engineer, so I might well be misunderstanding PyPy's current capability. For easy-to-use vector graphics output, like 1980s BASIC computers, I've settled on Pygame. CPython libraries that I've used for other reasons include Scipy, Matplotlib, PIL, CV2, and Kivy. From ginolampascione at gmx.com Sun Feb 22 10:53:25 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Sun, 22 Feb 2015 07:53:25 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From ginolampascione at gmx.com Sun Feb 22 10:54:05 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Sun, 22 Feb 2015 07:54:05 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From ginolampascione at gmx.com Sun Feb 22 10:54:41 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Sun, 22 Feb 2015 07:54:41 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <866dceda-c481-4d68-a6bb-4ea84c1bec14@googlegroups.com> -- redacted -- From pkpearson at nowhere.invalid Sun Feb 22 11:49:31 2015 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 22 Feb 2015 16:49:31 GMT Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements References: Message-ID: On Sat, 21 Feb 2015 14:46:26 -0500, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore they > are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 have > element 'M' in common, therefore they form the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. [snip] I recommend continuing to work on your statement of the problem until it is detailed, precise, and complete -- something along the lines of, "Given a set of sets, return a set of sets having the following properties: (1)... (2)..." This approach often brings to light logical problems in the loosely sketched requirements. It also produces the outline of a testing regimen to determine whether an implemented solution is correct. -- To email me, substitute nowhere->runbox, invalid->com. From buzzard at invalid.invalid Sun Feb 22 12:02:10 2015 From: buzzard at invalid.invalid (duncan smith) Date: Sun, 22 Feb 2015 17:02:10 +0000 Subject: Algorithm for Creating Supersets of Smaller Sets Based on Common Elements In-Reply-To: References: Message-ID: <54ea0ba0$0$23001$862e30e2@ngroups.net> On 21/02/15 19:46, TommyVee wrote: > Start off with sets of elements as follows: > > 1. A,B,E,F > 2. G,H,L,P,Q > 3. C,D,E,F > 4. E,X,Z > 5. L,M,R > 6. O,M,Y > > Note that sets 1, 3 and 4 all have the element 'E' in common, therefore > they are "related" and form the following superset: > > A,B,C,D,E,F,X,Z > > Likewise, sets 2 and 5 have the element 'L' in common, then set 5 and 6 > have element 'M' in common, therefore they form the following superset: > > G,H,L,M,O,P,Q,R,Y > > I think you get the point. As long as sets have at least 1 common > element, they combine to form a superset. Also "links" (common > elements) between sets may go down multiple levels, as described in the > second case above (2->5->6). Cycles thankfully, are not possible. > > BTW, the number of individual sets (and resultant supersets) will be > very large. > > I don't know where to start with this. I thought about some type of > recursive algorithm, but I'm not sure. I could figure out the Python > implementation easy enough, I'm just stumped on the algorithm itself. > > Anybody have an idea? > > Thanks, Tom Tom, You could construct a graph G such that there exists an edge {v,w} for each pair of items that appear in a common set. Each connected component will contain the nodes of one of the supersets you're looking for. That's unnecessarily expensive, so you can adopt a similar approach using trees. http://en.wikipedia.org/wiki/Disjoint-set_data_structure Construct a forest of trees (initially one tree for each item) and join pairs of trees until you have one tree for each of your supersets. For each set of size n you only need to consider n-1 joins. That will ensure that any pair of items that are in one of the sets are contained in a single tree. The find and union operations are amortized constant, so this should be efficient for your large numbers of sets. The union_find module can be found at https://github.com/DuncanSmith147/KVMS. Cheers. Duncan >>> import union_find >>> sets = (['A','B','E','F'], ['G','H','L','P','Q'], ['C','D','E','F'], ['E','X','Z'], ['L','M','R'], ['O','M','Y']) >>> uf = union_find.UnionFindTree() >>> for a_set in sets: for item in a_set: try: uf.add(item) except ValueError: pass n = a_set[0] for item in a_set[1:]: is_merged = uf.union(n, item) >>> from collections import defaultdict >>> res = defaultdict(list) >>> for item in uf: res[uf.find(item)].append(item) >>> res.values() [['G', 'H', 'M', 'L', 'O', 'Q', 'P', 'R', 'Y'], ['A', 'C', 'B', 'E', 'D', 'F', 'X', 'Z']] >>> From lac at openend.se Sun Feb 22 12:22:01 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 18:22:01 +0100 Subject: Future of Pypy? In-Reply-To: Message from Dave Farrance of "Sun, 22 Feb 2015 15:36:42 +0000." References: of "Sun, 22 Feb 2015 12:45:03 +0000." Message-ID: <201502221722.t1MHM16H001195@fido.openend.se> In a message of Sun, 22 Feb 2015 15:36:42 +0000, Dave Farrance writes: >Laura Creighton wrote: > >>I don't understand 'an interpreter rather than a JIT'. PyPy has a >>JIT, that sort of is the whole point. > >Yes. I meant that from my end-user, non-software-engineer perspective, it >looked as though CPython was proceeding with leaps and bounds and that >PyPy remained mostly a proof-of-concept after several years. Oh no, that was the state of the world about 8 years ago. PyPy works great, and more and more people are using it in production all the time. >But thanks for your description of the issues. So once the core issues >are sorted out, if enough people can be found to work on it, then >hopefully the library conversions will follow apace. Well, most libraries just run, of course. It's the ones that are written in C that cause most of the problems. One of the problems, from a pure Have-PyPy-Take-Over-the-World perspective is that as things were moving along quite merrily, the CPython core developers decided to invent Python 3.0. This meant that everybody on the planet, who wanted their library to work with Python 3.0 had to convert it to work there. There was, and still is, an enormous amount of resentment about this. For a lot of people, the perception was, and still is that the benefits of Python 3.x over Python 2.x was not worth breaking backwards compatibility. And there still are plenty of places whose plan is to use Python 2.7 indefinitely into the far future. I've got 15 years worth of commerical python code out there in the world, and nobody wants it converted enough to pay me to do so. Their position is that it runs quite well enough as it is. I'm sure not going to convert the stuff for fun. Practically every Python consultant on the planet is in the same boat. Things will get converted when there is a compelling business argument to do so, but not before, and for a lot of code the answer is never. Given the nature of this political problem, it is not surprising that the proponents of Python 3.0 spent a lot of effort talking about the benefits, and praising the people who converted their stuff, and making a huge effort in the public relations lines. The whole thing could have blown up in their faces, as is quite common when you decide to make the 'second version' of a language. It happened to Perl. So this creates buzz and warm feelings about Python 3.0. In contrast, on the PyPy team, there is nobody who doesn't consider public relations and marketing and 'creating the warm fuzzy feelings in the users' somewhere between 'unpleasant duty' and 'sheer torture'. The set of human skills you need to have to be good as this sort of thing is not a set that we have, either collectively or in individuals. We're much more into 'letting the code speak for itself', which, of course, it does not do. A lot of us, indeed were raised on a philosophy that it is morally wrong to try to influence people. You can give them options, and you can even explain the options that you are giving them, and you can argue with others in favour of certain options, but when it comes right down to it, everybody has to make their own decision. This is all well and virtuous, but the fact of the matter is that a large number of people aren't competant to make their own decisions, and even among those that are there exist a large number who very much don't want to do such a thing. If you are trying to get such people to adopt your software, you need to provide a completely different experience. They need to feel comfortable, and safe, and among a large community, and, well, I don't know what else they want. That is part of the problem. I am pretty sure that what they want is something that I never pay a lot of attention to. I mean I'm a charter member of the 'always-sacrifice-comfort-in-order-to-have-fun-and-interesting-times' club. And my marketing skills, such as they are, are much above average for the PyPy gang - though some members are learning a bit, slowly, through necessity. But you notice that we have only 1 blog, and things are added to it very slowly. There are people all over planet python who blog about things every week, for fun. There is no way we can compete with them. So, until some people with such skills decide to take an interest in PyPy, our marketing effort is going to limp. And I personally feel pretty bad about asking some poor soul who has just made his C extension work with 3.0 go back and do it _again_ in a more PyPy friendly way. But if Armin gets the Transactional Memory to be usable in a robust way, (as opposed to now where it is only a bit more than a proof of concept) then things could rocket off again. Because one thing we do know is that people who are completely and utterly ignorant about whether having multiple cores will improve their code still want to use a language that lets them use the multiple processors. If the TM dream of having that just happen, seemlessly (again, no promises) is proven to be true, well .... we think that the hordes will suddenly be interested in PyPy. But you never know. Lots of people were doing -complain-complain-python-is-too-slow, and wouldn't take the argument that developer time is worth more than CPU time and the like. We made them PyPy. They've stopped complaining about the speed, but they aren't using PyPy either. Seems they didn't really need the speed, after all. It is tempting to believe that they just liked complaining. So maybe the 'cannot use all my cores' complaint is fueled by other people who complain for the fun of it. Then there will be no hordes even when we crack that nut too. But it is very, very interesting and a whole lot of fun. Laura From pfranken85 at gmail.com Sun Feb 22 12:47:31 2015 From: pfranken85 at gmail.com (pfranken85 at gmail.com) Date: Sun, 22 Feb 2015 09:47:31 -0800 (PST) Subject: Question on asyncio Message-ID: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> Hello! I am just trying to get familiar with asyncio. It seems to be a good thing, however, I am still having troubles and feel pretty puzzled although I think I got the point what async IO means. This is the problem I am trying to accomplish: I have some functions which are reading values from hardware. If one of the values changes, I want a corresponding notification to the connected clients. The network part shouldn't be the problem. Here is what I got so far: @asyncio.coroutine def check(): old_val = read_value_from_device() yield from asyncio.sleep(2) new_val = read_value_from_device() # we may have fluctuations, so we introduce a threshold if abs(new_val-old_val) > 0.05: return new_val else: return None @asyncio.coroutine def runner(): while 1: new = yield from check() print(new) loop = asyncio.get_event_loop() loop.run_until_complete(update()) Is this the way one would accomplish this task? Or are there better ways? Should read_value_from_device() be a @coroutine as well? It may contain parts that take a while ... Of course, instead of print(new) I would add the corresponding calls for notifying the client about the update. Thanks! From luisjosenovoa at gmail.com Sun Feb 22 12:53:33 2015 From: luisjosenovoa at gmail.com (LJ) Date: Sun, 22 Feb 2015 09:53:33 -0800 (PST) Subject: id() and is operator Message-ID: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Hi everyone. Quick question here. Lets suppose if have the following numpy array: b=np.array([[0]*2]*3) and then: >>> id(b[0]) 45855552 >>> id(b[1]) 45857512 >>> id(b[2]) 45855552 Please correct me if I am wrong, but according to this b[2] and b[0] are the same object. Now, >>> b[0] is b[2] False Any clarification is much appreciated. Cheers, From lac at openend.se Sun Feb 22 13:13:31 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 19:13:31 +0100 Subject: id() and is operator In-Reply-To: Message from LJ of "Sun, 22 Feb 2015 09:53:33 -0800." <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Message-ID: <201502221813.t1MIDVpD011539@fido.openend.se> In a message of Sun, 22 Feb 2015 09:53:33 -0800, LJ writes: >Hi everyone. Quick question here. Lets suppose if have the following numpy array: > >b=np.array([[0]*2]*3) > >and then: > >>>> id(b[0]) >45855552 >>>> id(b[1]) >45857512 >>>> id(b[2]) >45855552 > >Please correct me if I am wrong, but according to this b[2] and b[0] are the same object. Now, > >>>> b[0] is b[2] >False You are running into one of the peculiarities of the python representation of numbers. It can make things more efficient to represent all common numbers as 'there is only one' of them. So. Python 2.7.9 (default, Dec 11 2014, 08:58:12) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> a = 1 >>> b = 1 >>> a is b True >>> a = 1001 >>> b = 1001 >>> a is b False -------- Don't rely on this. Other implementations are free to implement this however they like. -------- [PyPy 2.4.0 with GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>> a = 1001 >>>> b = 1001 >>>> a is b True Laura From subhabangalore at gmail.com Sun Feb 22 13:14:40 2015 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Sun, 22 Feb 2015 10:14:40 -0800 (PST) Subject: How to design a search engine in Python? In-Reply-To: References: <9701e262-7c29-49b6-bb66-8138f484bbea@googlegroups.com> <54e95e34$0$13006$c3e8da3$5496439d@news.astraweb.com> <0914bd64-6a5b-4014-af6e-f41dcb0c4fba@googlegroups.com> Message-ID: On Sunday, February 22, 2015 at 2:42:48 PM UTC+5:30, Laura Creighton wrote: > In a message of Sat, 21 Feb 2015 22:07:30 -0800, write > >Dear Sir, > > > >Thank you for your kind suggestion. Let me traverse one by one. > >My special feature is generally Semantic Search, but I am trying to build > >a search engine first and then go for semantic I feel that would give me a solid background to work around the problem. > > > >Regards, > >Subhabrata. > > You may find the API docs surrounding rdelbru.github.io/SIREn/ > of interest then. > > Laura Creighton Dear Madam, Thank you for your kind help. I would surely check then. Regards, Subhabrata. From jacob at blindza.co.za Sun Feb 22 13:29:15 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Sun, 22 Feb 2015 20:29:15 +0200 Subject: Accessible tools In-Reply-To: <20150221122918.7e4e39eb@bigbox.christie.dr> References: <479CFB67BB2640C09ADB16742A547172@JakesPC> <00DB5382-A26A-4157-898E-854C6C14CDF0@gmail.com> <20150221122918.7e4e39eb@bigbox.christie.dr> Message-ID: ----- Original Message ----- From: "Tim Chase" Subject: Re: Accessible tools > > While my experience has shown most of your items to be true, I'd > contend that > >>? Do not, have access to debugging tools. > > is mistaken or at least misinformed. For Python, I use the "pdb" > module all the time, and it's command-line driven. Combined with a > multi-terminal (whether multiple windows, virtual consoles, or a > tmux/screen session), I can easily bounce back and forth between a > "pdb" debugging session and the source code to make edits. Just to > check, I fired up the "yasr" terminal screen-reader, launched tmux > (using my quiet config, since it updates information on the screen > like the time on a regular basis, making it chatty), and stepped > through some Python code, checked variables, and walked up/down the > call-stack. I know most other languages have similar functionality > such as gdb for C code. Will check out PDB a bit more, but, honestly, my windows screenreader that use most of the time, jaws, doesn't always cooperate perfectly with command line/console interface - can be worked around, but, not all that easily at times - but, this page seems to offer enough detail relating to PDB, to start off with anyway: https://docs.python.org/3/library/pdb.html Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." From marfig at gmail.com Sun Feb 22 13:41:44 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Sun, 22 Feb 2015 19:41:44 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sat, 21 Feb 2015 12:22:58 +0000, Mark Lawrence wrote: > >Use your context manager at the outer level. > >import sqlite3 as lite > >try: > with lite.connect('data.db') as db: > try: > db.execute(sql, parms) > except lite.IntegrityError: > raise ValueError('invalid data') >except lite.DatabaseError: > raise OSError('database file corrupt or not found.') The sqlite context manager doesn't close a database connection on exit. It only ensures, commits and rollbacks are performed. From no.email at nospam.invalid Sun Feb 22 14:02:29 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 11:02:29 -0800 Subject: Future of Pypy? References: Message-ID: <87fv9xdb22.fsf@jester.gateway.pace.com> Laura Creighton writes: > Because one thing we do know is that people who are completely and > utterly ignorant about whether having multiple cores will improve > their code still want to use a language that lets them use the > multiple processors. If the TM dream of having that just happen, > seemlessly (again, no promises) is proven to be true, well .... we > think that the hordes will suddenly be interested in PyPy. TM is a useful feature but it's unlikely to be the thing that attracts "the hordes". More important is to eliminate the GIL and hopefully have lightweight (green) threads that can still run on multiple cores, like in GHC and Erlang. Having them communicate by mailboxes/queues is sufficient most of the time, and in Erlang it's the only method allowed in theory (there are some optimizations taking place behind the scenes). TM hasn't gotten that much uptake in GHC (one of the earliest HLL implementations of TM) in part because its performance cost is significant when there's contention. I wonder if Clojure programmers use it more. From breamoreboy at yahoo.co.uk Sun Feb 22 14:07:03 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 22 Feb 2015 19:07:03 +0000 Subject: try pattern for database connection with the close method In-Reply-To: References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On 22/02/2015 18:41, Mario Figueiredo wrote: > On Sat, 21 Feb 2015 12:22:58 +0000, Mark Lawrence > wrote: > >> >> Use your context manager at the outer level. >> >> import sqlite3 as lite >> >> try: >> with lite.connect('data.db') as db: >> try: >> db.execute(sql, parms) >> except lite.IntegrityError: >> raise ValueError('invalid data') >> except lite.DatabaseError: >> raise OSError('database file corrupt or not found.') > > The sqlite context manager doesn't close a database connection on > exit. It only ensures, commits and rollbacks are performed. > Where in the documentation does it state that? If it does, it certainly breaks my expectations, as I understood the whole point of Python context managers is to do the tidying up for you. Or have you misread what it says here https://docs.python.org/3/library/sqlite3.html#using-the-connection-as-a-context-manager ? >>> import sqlite3 >>> with sqlite3.connect(r'C:\Users\Mark\Documents\Cash\Data\cash.sqlite') as db: ... db.execute('select count(*) from accounts') ... >>> db.close() >>> Looks like you're correct. Knock me down with a feather, Clevor Trevor. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python.list at tim.thechases.com Sun Feb 22 14:14:24 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 22 Feb 2015 13:14:24 -0600 Subject: Accessible tools In-Reply-To: References: <479CFB67BB2640C09ADB16742A547172@JakesPC> <00DB5382-A26A-4157-898E-854C6C14CDF0@gmail.com> <20150221122918.7e4e39eb@bigbox.christie.dr> Message-ID: <20150222131424.6364c305@bigbox.christie.dr> On 2015-02-22 20:29, Jacob Kruger wrote: > jaws, doesn't always cooperate perfectly with command line/console > interface I've heard that on multiple occasions. Since I mostly work with Linux, the only terminal-with-screen-reader hints I've heard involve using TeraTerm as the SSH client with NVDA on Windows. Sorry I can't be of much more assistance there, since I haven't really used Windows for years. > this page seems to offer enough detail relating to PDB, to start > off with anyway: https://docs.python.org/3/library/pdb.html My generally process is as follows: 1) add the following line some place before where I want to debug: import pdb; pdb.set_trace() 2) run my program 3) when it hits that line of code, it drops you to the debugging prompt 4) poke around, using "print" followed by the thing(s) I want to inspect such as print dir(something) print my_var.some_field 5) step to the next line of code with "n" or step into a function/method call with "s" or exit/return from the current function/method with "r". I'll also use "l" (ell) to list some context around the current line so I can tell where I am in the source code. This is particularly helpful as I can use the corresponding line-number(s) to jump to the same line in my editor if I spot the offending line of code, then edit it. very rarely, I'll actually set additional breakpoints (optionally making them conditional) but I *always* have to look up the syntax for that. Hopefully that gets you at least to the point where debugging isn't some laborious task. Best wishes, -tkc From skip.montanaro at gmail.com Sun Feb 22 14:15:09 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 22 Feb 2015 13:15:09 -0600 Subject: try pattern for database connection with the close method In-Reply-To: References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sun, Feb 22, 2015 at 12:41 PM, Mario Figueiredo wrote: > The sqlite context manager doesn't close a database connection on > exit. It only ensures, commits and rollbacks are performed. Sorry, I haven't paid careful attention to this thread, so perhaps this has already been suggested, however... Can't you write your own class which delegates to the necessary sqlite3 bits and has a context manager with the desired behavior? Thinking out loud, you could define a ConnectionMgr class which accepts a sqlite3 connection as a parameter: class ConnectionMgr(object): def __init__(self, conn): self.conn = conn def __enter__(self): ... def __exit__(self, type, value, exception): if self.conn is not None: ... close self.conn connection here ... self.conn = None def __getattr__(self, attr): return getattr(self.conn, attr) then... try: with MyConnection(lite.connect('data.db')) as db: ... except lite.DatabaseError: ... Might also have to __enter__ and __exit__ self.conn as appropriate. Skip From gherron at digipen.edu Sun Feb 22 14:16:56 2015 From: gherron at digipen.edu (Gary Herron) Date: Sun, 22 Feb 2015 11:16:56 -0800 Subject: id() and is operator In-Reply-To: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Message-ID: <54EA2B28.8070101@digipen.edu> On 02/22/2015 09:53 AM, LJ wrote: > Hi everyone. Quick question here. Lets suppose if have the following numpy array: > > b=np.array([[0]*2]*3) > > and then: > >>>> id(b[0]) > 45855552 >>>> id(b[1]) > 45857512 >>>> id(b[2]) > 45855552 > > Please correct me if I am wrong, but according to this b[2] and b[0] are the same object. Now, > >>>> b[0] is b[2] > False > > > Any clarification is much appreciated. > > Cheers, In fact, b[0] and b[2] are different objects as can be seen here: >>> import numpy as np >>> b=np.array([[0]*2]*3) >>> b[0]=1 // broadcast into both ints in row 0 >>> b[1]=2 // ... row 1 >>> b[2]=3 // ... row 2 >>> b array([[1, 1], [2, 2], [3, 3]]) When you extracted b[0], you got a newly created python/numpy object (1x2 array of ints) briefly stored at location 45855552 but then deleted immediately after that use. A little later, the extraction of b[2] used the same bit of memory. The id of a temporarily created value is meaningless, and apparently misleading. As a separate issue, each of b, b[0], b[1], and b[2] do *all* refer to the same underlying array of ints as can be seen here: >>> r = b[0] >>> r[0] = 123 >>> b array([[123, 1], [ 2, 2], [ 3, 3]]) but the Python/numpy objects that wrap portions of that underlying array of ints are all distinct. Gary Herron -- Dr. Gary Herron Department of Computer Science DigiPen Institute of Technology (425) 895-4418 From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Sun Feb 22 14:20:44 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Sun, 22 Feb 2015 19:20:44 +0000 Subject: Future of Pypy? References: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> Message-ID: <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> Dave Farrance wrote: >Steven D'Aprano wrote: > >>I assume you're talking about drawing graphics rather than writing text. Can >>you tell us which specific library or libraries won't run under PyPy? > >Yes, mainly the graphics. I'm a hardware engineer, not a software >engineer, so I might well be misunderstanding PyPy's current capability. > >For easy-to-use vector graphics output, like 1980s BASIC computers, I've >settled on Pygame. CPython libraries that I've used for other reasons >include Scipy, Matplotlib, PIL, CV2, and Kivy. I see that PyPy's website says that PIL (Pillow) works. I have so far only used Python libraries that were readily available as binaries for Windows, or were already available in Linux distro repositories. In Ubuntu, for example, Pillow is available for CPython but not PyPy. Is there a guide to tell me (in non-developer language, hopefully) how to install Pillow for PyPy on Ubuntu? From rosuav at gmail.com Sun Feb 22 14:23:53 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 06:23:53 +1100 Subject: id() and is operator In-Reply-To: <201502221813.t1MIDVpD011539@fido.openend.se> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> <201502221813.t1MIDVpD011539@fido.openend.se> Message-ID: On Mon, Feb 23, 2015 at 5:13 AM, Laura Creighton wrote: > In a message of Sun, 22 Feb 2015 09:53:33 -0800, LJ writes: >>Hi everyone. Quick question here. Lets suppose if have the following numpy array: >> >>b=np.array([[0]*2]*3) >> >>and then: >> >>>>> id(b[0]) >>45855552 >>>>> id(b[1]) >>45857512 >>>>> id(b[2]) >>45855552 >> >>Please correct me if I am wrong, but according to this b[2] and b[0] are the same object. Now, >> >>>>> b[0] is b[2] >>False > > > You are running into one of the peculiarities of the python representation > of numbers. It can make things more efficient to represent all common > numbers as 'there is only one' of them. That shouldn't break the correspondence between id() and the is operator. The id function is documented as returning an integer which is "guaranteed to be unique among simultaneously existing objects", and if all three elements of b exist through the entire duration of this experiment, it should be perfectly safe to compare their id()s to check object identity. So the only explanation I can think of is: When you subscript a numpy array, you aren't getting back a reference to a pre-existing object, but you are instead getting a brand new object which is being created for you. (This theory is supported by a vague recollection that subscripting a numpy array returns a view of some sort, but you'd have to check the docs.) If that theory is correct, then you'd expect to find that the id() of such a thing is not stable; and that is, in fact, what I see: >>> import numpy as np >>> b=np.array([[0]*2]*3) >>> id(b[0]) 26806960 >>> id(b[0]) 26655344 >>> id(b[0]) 26820432 >>> id(b[0]) 26806960 >>> id(b[0]) 26655344 After a few iterations, they're getting reused, but it's not like playing with a Python list, where you would be getting back the exact same object every time You'd have to check the docs to be sure, but this is how I would go about exploring the situation. ChrisA From lac at openend.se Sun Feb 22 14:51:41 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 20:51:41 +0100 Subject: Future of Pypy? In-Reply-To: Message from Paul Rubin of "Sun, 22 Feb 2015 11:02:29 -0800." <87fv9xdb22.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> Message-ID: <201502221951.t1MJpfWr030860@fido.openend.se> In a message of Sun, 22 Feb 2015 11:02:29 -0800, Paul Rubin writes: >Laura Creighton writes: >> Because one thing we do know is that people who are completely and >> utterly ignorant about whether having multiple cores will improve >> their code still want to use a language that lets them use the >> multiple processors. If the TM dream of having that just happen, >> seemlessly (again, no promises) is proven to be true, well .... we >> think that the hordes will suddenly be interested in PyPy. > >TM is a useful feature but it's unlikely to be the thing that attracts >"the hordes". More important is to eliminate the GIL and hopefully have >lightweight (green) threads that can still run on multiple cores, like >in GHC and Erlang. Having them communicate by mailboxes/queues is >sufficient most of the time, and in Erlang it's the only method allowed >in theory (there are some optimizations taking place behind the scenes). >TM hasn't gotten that much uptake in GHC (one of the earliest HLL >implementations of TM) in part because its performance cost is >significant when there's contention. I wonder if Clojure programmers >use it more. The GIL isn't going away from PyPy any time real soon, alas. Armin has some pretty cool ideas about what to do about contention, but if you want to hear them, its better if you go post that to pypy-dev at python.org so you can get it from the man directly rather that hearing my paraphrase. Or ask away on the #pypy channel on freenode ... But this reminds me that I have to get Lennart Augustsson and Armin Rigo in the same room some time. Should be fun. Laura From lac at openend.se Sun Feb 22 14:58:12 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 20:58:12 +0100 Subject: id() and is operator In-Reply-To: Message from Gary Herron of "Sun, 22 Feb 2015 11:16:56 -0800." <54EA2B28.8070101@digipen.edu> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com><54EA2B28.8070101@digipen.edu> Message-ID: <201502221958.t1MJwCjk032182@fido.openend.se> Ooops, I missed the numpy, so I thought that it was the contents of the array that was causing the problem. My very bad. Apologies. Laura From no.email at nospam.invalid Sun Feb 22 15:14:45 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 12:14:45 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> Message-ID: <87bnkld7pm.fsf@jester.gateway.pace.com> Laura Creighton writes: > The GIL isn't going away from PyPy any time real soon, alas. I thought the GIL's main purpose was to avoid having to lock all the CPython refcount updates, so if PyPy has tracing GC, why is there still a GIL? And how is TM going to help with parallelism if the GIL is still there? > Armin has some pretty cool ideas about what to do about contention, > but if you want to hear them, its better if you go post that to > pypy-dev at python.org... Or ask away on the #pypy channel on freenode It would be nice if he blogged something about them. > But this reminds me that I have to get Lennart Augustsson and Armin > Rigo in the same room some time. Should be fun. I thought the STM stuff in GHC was done by the Simon's. Armin should certainly have Simon Marlow's book about concurrency and Haskell: http://chimera.labs.oreilly.com/books/1230000000929/index.html From ethan at stoneleaf.us Sun Feb 22 16:02:00 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 22 Feb 2015 13:02:00 -0800 Subject: Design thought for callbacks In-Reply-To: <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> Message-ID: <54EA43C8.2090309@stoneleaf.us> On 02/22/2015 05:13 AM, Cem Karan wrote: > Output: > From Evil Zombie: Surprise! > From Your Significant Other: Surprise! > > In this case, the user made an error (just as Marko said in his earlier message), > and forgot about the callback he registered with the library. The callback isn't > really rising from the dead; as you say, either its been garbage collected, or it > hasn't been. However, you may not be ready for a callback to be called at that > moment in time, which means you're surprised by unexpected behavior. But the unexpected behavior is not a problem with Python, nor with your library -- it's a bug in the fellow-programmer's code, and you can't (or at least shouldn't) try to prevent those kinds of bugs from manifesting -- they'll just get bitten somewhere else by the same bug. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From cfkaran2 at gmail.com Sun Feb 22 16:17:21 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 16:17:21 -0500 Subject: Design thought for callbacks In-Reply-To: <54EA43C8.2090309@stoneleaf.us> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> Message-ID: On Feb 22, 2015, at 4:02 PM, Ethan Furman wrote: > On 02/22/2015 05:13 AM, Cem Karan wrote: > >> Output: >> From Evil Zombie: Surprise! >> From Your Significant Other: Surprise! >> >> In this case, the user made an error (just as Marko said in his earlier message), >> and forgot about the callback he registered with the library. The callback isn't >> really rising from the dead; as you say, either its been garbage collected, or it >> hasn't been. However, you may not be ready for a callback to be called at that >> moment in time, which means you're surprised by unexpected behavior. > > But the unexpected behavior is not a problem with Python, nor with your library -- it's a bug in the fellow-programmer's > code, and you can't (or at least shouldn't) try to prevent those kinds of bugs from manifesting -- they'll just get > bitten somewhere else by the same bug. I agree with you, but until a relatively new programmer has gotten used to what callbacks are and what they imply, I want to make things easy. For example, if the API subclasses collections.abc.MutableSet, and the documentation states that you can only add callbacks to this particular type of set, then a new programmer will naturally decide that either a) they need to dispose of the set, and if that isn't possible, then b) they need to delete their callback from the set. It won't occur to them that their live object will just magically 'go away'; its a member of a set! My goal is to make things as pythonic (whatever that means in this case) and obvious as possible. Ideally, a novice can more or less guess what will happen with my API without really having to read the documentation on it. Thanks, Cem Karan From marko at pacujo.net Sun Feb 22 16:21:42 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 23:21:42 +0200 Subject: Question on asyncio References: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> Message-ID: <8761atiqvt.fsf@elektro.pacujo.net> pfranken85 at gmail.com: > I have some functions which are reading values from hardware. If one > of the values changes, I want a corresponding notification to the > connected clients. The network part shouldn't be the problem. Here is > what I got so far: > > @asyncio.coroutine > def check(): > old_val = read_value_from_device() > yield from asyncio.sleep(2) > new_val = read_value_from_device() > # we may have fluctuations, so we introduce a threshold > if abs(new_val-old_val) > 0.05: > return new_val > else: > return None > > @asyncio.coroutine > def runner(): > while 1: > new = yield from check() > print(new) In asyncio, you typically ignore the value returned by yield. While generators use yield to communicate results to the calling program, coroutines use yield only as a "trick" to implement cooperative multitasking and an illusion of multithreading. Thus, "yield from" in asyncio should be read, "this is a blocking state." > Is this the way one would accomplish this task? Or are there better > ways? Should read_value_from_device() be a @coroutine as well? It may > contain parts that take a while ... Of course, instead of print(new) I > would add the corresponding calls for notifying the client about the > update. How do you read a value from the hardware? Do you use a C extension? Do you want read_value_from_device() to block until the hardware has the value available or is the value always available for instantaneous reading? If the value is available instantaneously, you don't need to turn it into a coroutine. However, if blocking is involved, you definitely should do that. Depending on your hardware API it can be easy or difficult. If you are running CPython over linux, hardware access probably is abstracted over a file descriptor and a coroutine interface would be simple. Marko From marko at pacujo.net Sun Feb 22 16:25:12 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 23:25:12 +0200 Subject: id() and is operator References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Message-ID: <871tlhiqpz.fsf@elektro.pacujo.net> LJ : >>>> id(b[0]) > 45855552 [...] >>>> id(b[2]) > 45855552 > > Please correct me if I am wrong, but according to this b[2] and b[0] > are the same object. Now, > >>>> b[0] is b[2] > False This is a true statement: If X is Y, then id(X) == id(Y). However, this is generally not a true statement: If X is Y, then id(X) is id(Y). Marko From marko at pacujo.net Sun Feb 22 16:34:30 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 22 Feb 2015 23:34:30 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> Message-ID: <87wq39hbq1.fsf@elektro.pacujo.net> Cem Karan : > My goal is to make things as pythonic (whatever that means in this > case) and obvious as possible. Ideally, a novice can more or less > guess what will happen with my API without really having to read the > documentation on it. If you try to shield your user from the complexities of asynchronous programming, you will only cause confusion. You will definitely need to document all nooks and crannies of the semantics of the callback API and your user will have to pay attention to every detail of your spec. Your user, whether novice or an expert, will thank you for your unambiguous specification even if it is complicated. Marko From rosuav at gmail.com Sun Feb 22 16:36:22 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 08:36:22 +1100 Subject: id() and is operator In-Reply-To: <871tlhiqpz.fsf@elektro.pacujo.net> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> <871tlhiqpz.fsf@elektro.pacujo.net> Message-ID: On Mon, Feb 23, 2015 at 8:25 AM, Marko Rauhamaa wrote: > This is a true statement: > > If X is Y, then id(X) == id(Y). > > However, this is generally not a true statement: > > If X is Y, then id(X) is id(Y). Irrelevant, because the identities of equal integers didn't come into this. ChrisA From lac at openend.se Sun Feb 22 16:45:48 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 22:45:48 +0100 Subject: Future of Pypy? In-Reply-To: Message from Dave Farrance of "Sun, 22 Feb 2015 19:20:44 +0000." <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> References: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> Message-ID: <201502222145.t1MLjmQc021012@fido.openend.se> Good news -- it seems to be working fine with PyPy. https://travis-ci.org/hugovk/Pillow/builds for me, not extensively tested, it just seems to be working. I have several pypy's floating around here, each within its own virtualenv. If you aren't familiar with virtualenv, read all about it here: http://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ Note the first question to the blog writer is 'how to get it to work with pypy'. Do what he says. virtualenv -p /path/to/pypy env but, if you want to use more bleeding edge pypy you will want: # from a tarball $ virtualenv -p /opt/pypy-c-jit-41718-3fb486695f20-linux/bin/pypy my-pypy-env # from the mercurial checkout $ virtualenv -p /path/to/pypy/pypy/translator/goal/pypy-c my-pypy-env I've only got bleeding edge PyPys around here, in virtualenvs, but in all of them import sys from PIL import Image for infile in sys.argv[1:]: try: with Image.open(infile) as im: print(infile, im.format, "%dx%d" % im.size, im.mode) except IOError: pass which I pasted right in from http://pillow.readthedocs.org/en/latest/handbook/tutorial.html seems to be working just fine for me. Hardly an exhaustive test, but ... well, try it and see how it goes for you. I don't know what time it is where you are, but it is 22:44 here now, and alas I promised a kivy demo to a client tomorrow morning, and, double alas, I haven't written it yet. It shouldn't take more than an hour or three to write, but I am going to have to stop having pleasant chats about pypy for a while and get this thing done ... :) Laura From jkuplinsky at gmail.com Sun Feb 22 17:06:44 2015 From: jkuplinsky at gmail.com (jkuplinsky at gmail.com) Date: Sun, 22 Feb 2015 14:06:44 -0800 (PST) Subject: calling subprocess Message-ID: Hi, I thought this would be easy: for subprocess import call call (['cd', r'C:\apps'], shell = True) It doesn't work -- tried with/without prefix r, escaped backslashes, triple quotes, str(), .. nothing seems to work (it doesn't complain, but it doesn't change directories either) -- what am I doing wrong? Thanks. JK From cfkaran2 at gmail.com Sun Feb 22 17:09:01 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Sun, 22 Feb 2015 17:09:01 -0500 Subject: Design thought for callbacks In-Reply-To: <87wq39hbq1.fsf@elektro.pacujo.net> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net> Message-ID: <0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> On Feb 22, 2015, at 4:34 PM, Marko Rauhamaa wrote: > Cem Karan : > >> My goal is to make things as pythonic (whatever that means in this >> case) and obvious as possible. Ideally, a novice can more or less >> guess what will happen with my API without really having to read the >> documentation on it. > > If you try to shield your user from the complexities of asynchronous > programming, you will only cause confusion. You will definitely need to > document all nooks and crannies of the semantics of the callback API and > your user will have to pay attention to every detail of your spec. > > Your user, whether novice or an expert, will thank you for your > unambiguous specification even if it is complicated. Documentation is a given; it MUST be there. That said, documenting something, but still making it surprising, is a bad idea. For example, several people have been strongly against using a WeakSet to hold callbacks because they expect a library to hold onto callbacks. If I chose not to do that, and used a WeakSet, then even if I documented it, it would still end up surprising people (and from the sound of it, more people would be surprised than not). Thanks, Cem Karan From lac at openend.se Sun Feb 22 17:13:56 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 23:13:56 +0100 Subject: Future of Pypy? In-Reply-To: Message from Paul Rubin of "Sun, 22 Feb 2015 12:14:45 -0800." <87bnkld7pm.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <87bnkld7pm.fsf@jester.gateway.pace.com> Message-ID: <201502222213.t1MMDuba026550@fido.openend.se> In a message of Sun, 22 Feb 2015 12:14:45 -0800, Paul Rubin writes: >Laura Creighton writes: >> The GIL isn't going away from PyPy any time real soon, alas. > >I thought the GIL's main purpose was to avoid having to lock all the >CPython refcount updates, so if PyPy has tracing GC, why is there still >a GIL? And how is TM going to help with parallelism if the GIL is still >there? This requires a long answer. a very long answer. More later, I must work this evening. >> Armin has some pretty cool ideas about what to do about contention, >> but if you want to hear them, its better if you go post that to >> pypy-dev at python.org... Or ask away on the #pypy channel on freenode > >It would be nice if he blogged something about them. You are asking for water to roll up-hill. If you want the joy of hearing the cool ideas as Armin has them, you need to hang out on the irc channel. Of course, if you are interested in such things this makes hanging out there worthwhile. >> But this reminds me that I have to get Lennart Augustsson and Armin >> Rigo in the same room some time. Should be fun. > >I thought the STM stuff in GHC was done by the Simon's. Armin should >certainly have Simon Marlow's book about concurrency and Haskell: Of course, but if you think that Lennart Augustsson is not familiar with every aspect of every Haskell compiler on the planet .... well then I know Lennart better than you do. And given that Lennart is a friend, well really a good friend of my lover and a something-better- than-an-acquaintance with me ---- I should make the effort to get these two under the same roof (mine, by preference) for the fun of the experience. So thank you for giving me this idea ... Laura From mail at timgolden.me.uk Sun Feb 22 17:14:24 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Sun, 22 Feb 2015 22:14:24 +0000 Subject: calling subprocess In-Reply-To: References: Message-ID: <54EA54C0.3060109@timgolden.me.uk> On 22/02/2015 22:06, jkuplinsky at gmail.com wrote: > Hi, > > I thought this would be easy: > > > for subprocess import call call (['cd', r'C:\apps'], shell = True) > > > It doesn't work -- tried with/without prefix r, escaped backslashes, > triple quotes, str(), .. nothing seems to work (it doesn't complain, > but it doesn't change directories either) -- what am I doing wrong? Two things: 1) All you're doing is running up a subprocess, changing directory *within it* and then closing it. (Altho' good job spotting that you'd need shell=True if you did indeed want to do what you're doing here). 2) Use os.chdir: that's what it's there for TJG From rosuav at gmail.com Sun Feb 22 17:21:54 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 09:21:54 +1100 Subject: calling subprocess In-Reply-To: References: Message-ID: On Mon, Feb 23, 2015 at 9:06 AM, wrote: > I thought this would be easy: > > > for subprocess import call > call (['cd', r'C:\apps'], shell = True) > > > It doesn't work -- tried with/without prefix r, escaped backslashes, triple quotes, str(), .. nothing seems to work (it doesn't complain, but it doesn't change directories either) -- what am I doing wrong? It does work. But what it does is spawn a shell, change the working directory _of that shell_, and then terminate it. The working directory change won't apply to your process. It sounds to me like you're adopting a "shotgun debugging" [1] approach. If you don't know what your changes are going to do, why are you making them? I recommend, instead, a policy of examination and introspection - what I tend to refer to as IIDPIO debugging: If In Doubt, Print It Out. The first argument to subprocess.call() is a list of parameters; you can assign that to a name and print it out before you do the call: from subprocess import call args = ['cd', r'C:\apps'] print(args) call(args, shell=True) Now do all your permutations of args, and see what effect they have. If the printed-out form is identical, there's no way it can affect the subprocess execution. Side point: *Copy and paste* your code rather than retyping it. The keyword here is "from", not "for", and if we can't depend on the code you're showing us, how can we recognize whether or not a bug exists in your real code? ChrisA [1] http://www.catb.org/jargon/html/S/shotgun-debugging.html From lac at openend.se Sun Feb 22 17:29:07 2015 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Feb 2015 23:29:07 +0100 Subject: Design thought for callbacks In-Reply-To: Message from Cem Karan of "Sun, 22 Feb 2015 17:09:01 -0500." <0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net><0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> Message-ID: <201502222229.t1MMT7Dn029607@fido.openend.se> In a message of Sun, 22 Feb 2015 17:09:01 -0500, Cem Karan writes: >Documentation is a given; it MUST be there. That said, documenting >something, but still making it surprising, is a bad idea. For >example, several people have been strongly against using a WeakSet to >hold callbacks because they expect a library to hold onto callbacks. >If I chose not to do that, and used a WeakSet, then even if I >documented it, it would still end up surprising people (and from the >sound of it, more people would be surprised than not). >Thanks, Cem Karan No matter what you do, alas, will surprise the hell out of people because callbacks do not behave as people expect. Among people who have used callbacks, what you are polling is 'what are people familiar with', and it seems for the people around here, now, WeakSets are not what they are familiar with. But that is not so surprising. How many people use WeakSets for _anything_? I've never used them, aside from 'ooh! cool shiny new language feature! Let's kick it around the park!' That people aren't familiar with WeakSets doesn't mean all that much. The question I have is does this architecture make things harder, easier or about the same to debug? To write tests for? to do Test Driven Design with? Laura From rosuav at gmail.com Sun Feb 22 17:41:04 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 09:41:04 +1100 Subject: Design thought for callbacks In-Reply-To: <201502222229.t1MMT7Dn029607@fido.openend.se> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net> <0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> <201502222229.t1MMT7Dn029607@fido.openend.se> Message-ID: On Mon, Feb 23, 2015 at 9:29 AM, Laura Creighton wrote: > But that is not so surprising. How many people use WeakSets for > _anything_? I've never used them, aside from 'ooh! cool shiny > new language feature! Let's kick it around the park!' That people > aren't familiar with WeakSets doesn't mean all that much. I haven't used weak *sets*, but I've used weak *mappings* on occasion. It's certainly not a common thing, but they have their uses. I have a MUD which must guarantee that there be no more than one instance of any given room (identified by a string that looks like a Unix path), but which will, if it can, flush rooms out of memory when nothing refers to them. So it has a mapping from the path strings to the instances, but with weak refs for the instances; if anything else is referring to that instance (eg a player character in the room), it'll hang around, and any time anyone else needs that room, they'll get the same instance back from the mapping; but any time the garbage collector notices that a room can be disposed of, it will be. Definitely not common though. ChrisA From marfig at gmail.com Sun Feb 22 18:25:26 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 23 Feb 2015 00:25:26 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sun, 22 Feb 2015 19:07:03 +0000, Mark Lawrence wrote: > >Looks like you're correct. Knock me down with a feather, Clevor Trevor. It took me by surprise when I first encountered it too. The rationale apparently is that the context manager is strictly a transactional feature, allowing for multiple context managers within the same connection to properly perform commits and rollbacks on multiple transactions. From marfig at gmail.com Sun Feb 22 18:30:13 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 23 Feb 2015 00:30:13 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sun, 22 Feb 2015 13:15:09 -0600, Skip Montanaro wrote: > >Sorry, I haven't paid careful attention to this thread, so perhaps >this has already been suggested, however... Can't you write your own >class which delegates to the necessary sqlite3 bits and has a context >manager with the desired behavior? Thinking out loud, you could define >a ConnectionMgr class which accepts a sqlite3 connection as a >parameter Indeed I could. Thank you. From marfig at gmail.com Sun Feb 22 18:35:41 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Mon, 23 Feb 2015 00:35:41 +0100 Subject: try pattern for database connection with the close method References: <6trfeate2ppvm1mcapgr0g4g2fd3vceab6@4ax.com> Message-ID: On Sat, 21 Feb 2015 16:22:36 +0100, Peter Otten <__peter__ at web.de> wrote: > >Why would you care about a few lines? You don't repeat them, do you? Put the >code into a function or a context manager and invoke it with Thanks for the suggestions that followed. From jkuplinsky at gmail.com Sun Feb 22 20:13:53 2015 From: jkuplinsky at gmail.com (jkuplinsky at gmail.com) Date: Sun, 22 Feb 2015 17:13:53 -0800 (PST) Subject: calling subprocess In-Reply-To: References: Message-ID: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> On Sunday, February 22, 2015 at 5:22:24 PM UTC-5, Chris Angelico wrote: > On Mon, Feb 23, 2015 at 9:06 AM, wrote: > > I thought this would be easy: > > > > > > for subprocess import call > > call (['cd', r'C:\apps'], shell = True) > > > > > > It doesn't work -- tried with/without prefix r, escaped backslashes, triple quotes, str(), .. nothing seems to work (it doesn't complain, but it doesn't change directories either) -- what am I doing wrong? > > It does work. But what it does is spawn a shell, change the working > directory _of that shell_, and then terminate it. The working > directory change won't apply to your process. > > It sounds to me like you're adopting a "shotgun debugging" [1] > approach. If you don't know what your changes are going to do, why are > you making them? I recommend, instead, a policy of examination and > introspection - what I tend to refer to as IIDPIO debugging: If In > Doubt, Print It Out. The first argument to subprocess.call() is a list > of parameters; you can assign that to a name and print it out before > you do the call: > > from subprocess import call > args = ['cd', r'C:\apps'] > print(args) > call(args, shell=True) > > Now do all your permutations of args, and see what effect they have. > If the printed-out form is identical, there's no way it can affect the > subprocess execution. > > Side point: *Copy and paste* your code rather than retyping it. The > keyword here is "from", not "for", and if we can't depend on the code > you're showing us, how can we recognize whether or not a bug exists in > your real code? > > ChrisA > > [1] http://www.catb.org/jargon/html/S/shotgun-debugging.html OK (1) sorry about for/from (2) print() sounds nice, but fact is , no matter what I try, i always get C:\\apps instead of c:\apps. So in this sense print() doesn't help much. Obviously i'm doing something wrong -- which is what you perhaps call shotgun debugging; but that's why i'm asking. From steve+comp.lang.python at pearwood.info Sun Feb 22 20:18:43 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 12:18:43 +1100 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> Message-ID: <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> Paul Rubin wrote: > Laura Creighton writes: >> Because one thing we do know is that people who are completely and >> utterly ignorant about whether having multiple cores will improve >> their code still want to use a language that lets them use the >> multiple processors. If the TM dream of having that just happen, >> seemlessly (again, no promises) is proven to be true, well .... we >> think that the hordes will suddenly be interested in PyPy. > > TM is a useful feature but it's unlikely to be the thing that attracts > "the hordes". More important is to eliminate the GIL *rolls eyes* I'm sorry, but the instant somebody says "eliminate the GIL", they lose credibility with me. Yes yes, I know that in *your* specific case you've done your research and (1) multi-threaded code is the best solution for your application and (2) alternatives aren't suitable. Writing multithreaded code is *hard*. It is not a programming model which comes naturally to most human beings. Very few programs are inherently parallelizable, although many programs have *parts* which can be successfully parallelized. I think that for many people, "the GIL" is just a bogeyman, or is being blamed for their own shortcomings. To take an extreme case, if you're running single-thread code on a single-core machine and still complaining about the GIL, you have no clue. (That's not *you personally* Paul, it's a generic "you".) There are numerous alternatives for those who are genuinely running into GIL-related issues. Jeff Knupp has a good summary: http://www.jeffknupp.com/blog/2013/06/30/pythons-hardest-problem-revisited/ One alternative that he misses is that for some programs, the simplest way to speed it up is to vectorize the core parts of your code by using numpy. No threads needed. For those who think that the GIL and the GIL alone is the problem, consider that Jython is nearly as old as CPython, it goes back at least 15 years. IronPython has been around for a long time too, and is possibly faster than CPython even in single threaded code. Neither has a GIL. Both are mature implementations, built on well-known, powerful platforms with oodles of business credibility (the JVM and .Net). IronPython even has the backing of Microsoft, it is one of the few non-Microsoft languages with a privileged position in the .Net ecosystem. Where are the people flocking to use Jython and IronPython? In fairness, there are good reasons why some people cannot use Jython or IronPython, or one of the other alternatives. But that demonstrates that the problem is more complex than just "the GIL". For removal of the GIL to really make a difference: - you must have at least two cores (that, at least, applies to most people these days); - you must be performing a task which is parallelizable and not inherently sequential (no point using multiple threads if each thread spends all its time waiting for the previous thread); - the task must be one that moving to some other multi-processing model (such as greenlets, multiprocess, etc.) is infeasible; - you must actually use multiple threads, and use them properly (no busy wait loops); - your threading bottleneck must be primarily CPU-bound, not I/O bound (CPython's threads are already very effective at parallelising I/O tasks); - and you must be using libraries and tools which prevent you moving to Jython or IronPython or some other alternative. I can't help but feel that the set of people for whom removal of the GIL would actually help is much smaller than, and different to, the set of people who complain about the GIL. -- Steven From rosuav at gmail.com Sun Feb 22 20:31:05 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 12:31:05 +1100 Subject: calling subprocess In-Reply-To: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> References: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> Message-ID: On Mon, Feb 23, 2015 at 12:13 PM, wrote: > (2) print() sounds nice, but fact is , no matter what I try, i always get C:\\apps instead of c:\apps. So in this sense print() doesn't help much. Obviously i'm doing something wrong -- which is what you perhaps call shotgun debugging; but that's why i'm asking. > Actually, that means it's helping a lot: it's showing you that, no matter what you fiddle with in terms of string literals, the resulting string is exactly the same. That's the point of printing stuff out :) ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 22 20:31:25 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 12:31:25 +1100 Subject: id() and is operator References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Message-ID: <54ea82ef$0$13003$c3e8da3$5496439d@news.astraweb.com> LJ wrote: > Hi everyone. Quick question here. Lets suppose if have the following numpy > array: > > b=np.array([[0]*2]*3) > > and then: > >>>> id(b[0]) > 45855552 >>>> id(b[1]) > 45857512 >>>> id(b[2]) > 45855552 > > Please correct me if I am wrong, but according to this b[2] and b[0] are > the same object. Not necessarily. CPython (the version of Python you are using) can reuse object IDs. This is not the case for all Pythons, e.g. Jython and IronPython never reuse IDs. That means that if you compare the ID of two objects in CPython which are not alive at the same time, they might have received the same ID. py> id("hello world") 3083591616 py> id("now what???") 3083591616 IDs are only unique if the objects are alive at the same time. Numpy arrays are effectively C arrays of low-level machine values, what Java calls "unboxed" values. So when you index a specific value, Python has to create a new object to hold it. (In this case, that object is also an array.) If that object is then garbage collected, the next time you ask for the value at an index, the freshly created object may end up with the same ID just by chance. py> import numpy as np py> b = np.array([[0]*2]*3) py> x = b[0] py> y = b[1] py> print id(x), id(y) 155749968 156001664 py> print id(b[0]), id(b[1]) # temporary objects that are thrown away 156055016 156055016 If you try it yourself, you may or may not get exactly the same results. You may need to print the IDs repeatedly until, just by chance, you end up with identical IDs. -- Steven From no.email at nospam.invalid Sun Feb 22 21:04:09 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 18:04:09 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87zj85bcyu.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > I'm sorry, but the instant somebody says "eliminate the GIL", they lose > credibility with me. Yes yes, I know that in *your* specific case you've > done your research and (1) multi-threaded code is the best solution for > your application and (2) alternatives aren't suitable. I don't see what the big deal is. I hear tons of horror stories about threads and I believe them, but the thing is, they almost always revolve around acquiring and releasing locks in the wrong order, forgetting to lock things, stuff like that. So I've generally respected the terror and avoided programming in that style, staying with a message passing style that may take an efficiency hit but seems to avoid almost all those problems. TM also helps with lock hazards and it's a beautiful idea--I just haven't had to use it yet. The Python IRC channel seems to rage against threads and promote Twisted for concurrency, but Twisted has always reminded me of Java. I use threads in Python all the time and haven't gotten bitten yet. > Writing multithreaded code is *hard*. It is not a programming model > which comes naturally to most human beings. Very few programs are > inherently parallelizable, although many programs have *parts* which > can be successfully parallelized. Parallel algorithms are complicated and specialized but tons of problems amount to "do the same thing with N different pieces of data", so-called embarassingly parallel. The model is you have a bunch of worker threads reading off a queue and processing the items concurrently. Sometimes separate processes works equally well, other times it's good to have some shared data in memory instead of communicating through sockets. If the data is mutable then have one thread own it and access it only with message passing, Erlang style. If it's immutable after initialization (protect it with a condition variable til initialization finishes) then you can have read-only access from anywhere. > if you're running single-thread code on a single-core machine and > still complaining about the GIL, you have no clue. Even the Raspberry Pi has 4 cores now, and fancy smartphones have had them for years. Single core cpu's are specialized and/or historical. > for some programs, the simplest way to speed it up is to vectorize the > core parts of your code by using numpy. No threads needed. Nice for numerical codes, not so hot for anything else. > Where are the people flocking to use Jython and IronPython? Shrug, who knows, those implementations were pretty deficient from what heard. > For removal of the GIL to really make a difference: ... > - you must be performing a task which is parallelizable and not inherently > sequential (no point using multiple threads if each thread spends all its > time waiting for the previous thread); That's most things involving concurrency these days. > - the task must be one that moving to some other multi-processing model > (such as greenlets, multiprocess, etc.) is infeasible; I don't understand this--there can be multiple ways to solve a problem. > - your threading bottleneck must be primarily CPU-bound, not I/O bound > (CPython's threads are already very effective at parallelising I/O tasks); If your concurrent program's workload makes it cpu-bound even 1% of the time, then you gain something by having it use your extra cores at those moments, instead of having those cores always do nothing. > - and you must be using libraries and tools which prevent you moving to > Jython or IronPython or some other alternative. I don't get this at all. Why should I not want Python to have the same capabilities? From rosuav at gmail.com Sun Feb 22 21:16:21 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 13:16:21 +1100 Subject: Future of Pypy? In-Reply-To: <87zj85bcyu.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> Message-ID: On Mon, Feb 23, 2015 at 1:04 PM, Paul Rubin wrote: >> if you're running single-thread code on a single-core machine and >> still complaining about the GIL, you have no clue. > > Even the Raspberry Pi has 4 cores now, and fancy smartphones have had > them for years. Single core cpu's are specialized and/or historical. Or virtual. I have a quad-core + hyperthreading CPU, but most of my VMs (in fact, all except the one that runs a Python buildbot, I think!) are restricted to a single core. If you rent a basic cheap machine from a cloud provider, you'll quite possibly be getting a single core, too. ChrisA From rosuav at gmail.com Sun Feb 22 21:29:28 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 13:29:28 +1100 Subject: Unrecognized backslash escapes in string literals Message-ID: In Python, unrecognized escape sequences are treated literally, without (as far as I can tell) any sort of warning or anything. This can mask bugs, especially when Windows path names are used: >>> 'C:\sqlite\Beginner.db' 'C:\\sqlite\\Beginner.db' >>> 'c:\sqlite\beginner.db' 'c:\\sqlite\x08eginner.db' To a typical Windows user, the two strings should be equivalent - case insensitive file names, who cares whether you say "Beginner" or "beginner"? But to Python, one of them will happen to work, the other will fail badly. Why is it that Python interprets them this way, and doesn't even give a warning? What happened to errors not passing silently? Or, looking at this the other way: Is there a way to enable such warnings/errors? I can't see one in 'python[3] -h', but if there's some way elsewhere, that would be a useful thing to recommend to people (I already recommend running Python 2 with -tt). ChrisA From davea at davea.name Sun Feb 22 21:38:23 2015 From: davea at davea.name (Dave Angel) Date: Sun, 22 Feb 2015 21:38:23 -0500 Subject: calling subprocess In-Reply-To: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> References: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> Message-ID: <54EA929F.70509@davea.name> On 02/22/2015 08:13 PM, jkuplinsky at gmail.com wrote: > > OK (1) sorry about for/from That's not what you should be sorry about. You should be sorry you didn't use cut&paste. > (2) print() sounds nice, but fact is , no matter what I try, i always get C:\\apps instead of c:\apps. So in this sense print() doesn't help much. Obviously i'm doing something wrong -- which is what you perhaps call shotgun debugging; but that's why i'm asking. > You probably are getting confused about the difference between str() and repr(). If you print the repr() of a string, it'll add quotes around it, and escape the unprintable codes. So it'll double the backslash. It also turns a newline into \n, and tabs into \t, and so on. Very useful. That's also what happens when you print a list that contains strings. The individual elements of the list are converted using repr(). Watch for the quotes to get a strong hint about what you're seeing. If you don't get a positive handle on how string literals relate to string variables, and on str() and repr(), and print(), you'll be jumping around the problem instead of solving it. Back to your original problem, which had you trying to use subprocess.call to change the current directory. Current directory is effectively (or actually) depending on the OS involved) an environment variable, and changes made in a child process are not magically returned to the parent. But even though there is an os.chdir() in Python, you really shouldn't use it. Long experience of many people show that you're better off manipulating the directories you need explicitly, converting any directory that's relative to something other than the current one, to an absolute. -- DaveA From ben+python at benfinney.id.au Sun Feb 22 21:41:26 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 23 Feb 2015 13:41:26 +1100 Subject: Unrecognized backslash escapes in string literals References: Message-ID: <85vbit73jd.fsf@benfinney.id.au> Chris Angelico writes: > In Python, unrecognized escape sequences are treated literally, > without (as far as I can tell) any sort of warning or anything. Right. Text strings literals are documented to work that way , which refers the reader to the language reference . > Why is it that Python interprets them this way, and doesn't even give > a warning? Because the interpretation of those literals is unambiguous and correct. It's unfortunate that MS Windows inherited the incompatible ?backslash is a path separator?, long after backslash was already established in many programming languages as the escape character. > Is there a way to enable such warnings/errors? A warning or error for a correctly formatted literal with an unambiguous meaning would be an up-Pythonic thing to have. I can see the motivation, but really the best solution is to learn that the backslash is an escape character in Python text string literals. This has the advantage that it's the same escape character used for text string literals in virtually every other programming language, so you're not needing to learn anything unusual. -- \ ?The deepest sin against the human mind is to believe things | `\ without evidence.? ?Thomas Henry Huxley, _Evolution and | _o__) Ethics_, 1893 | Ben Finney From no.email at nospam.invalid Sun Feb 22 21:45:07 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 18:45:07 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <87bnkld7pm.fsf@jester.gateway.pace.com> Message-ID: <87sidxbb2k.fsf@jester.gateway.pace.com> Laura Creighton writes: > And given that Lennart is a friend, well really a good friend of my > lover and a something-better- than-an-acquaintance with me ---- I > should make the effort to get these two under the same roof (mine, by > preference) for the fun of the experience. Oh cool, right, I forgot that you are in Sweden where he is. I've never met him but he's pretty visible online and yes, he sure knows a lot about Haskell. From davea at davea.name Sun Feb 22 21:47:27 2015 From: davea at davea.name (Dave Angel) Date: Sun, 22 Feb 2015 21:47:27 -0500 Subject: Unrecognized backslash escapes in string literals In-Reply-To: References: Message-ID: <54EA94BF.7080006@davea.name> On 02/22/2015 09:29 PM, Chris Angelico wrote: > In Python, unrecognized escape sequences are treated literally, > without (as far as I can tell) any sort of warning or anything. This > can mask bugs, especially when Windows path names are used: > >>>> 'C:\sqlite\Beginner.db' > 'C:\\sqlite\\Beginner.db' >>>> 'c:\sqlite\beginner.db' > 'c:\\sqlite\x08eginner.db' > > To a typical Windows user, the two strings should be equivalent - case > insensitive file names, who cares whether you say "Beginner" or > "beginner"? But to Python, one of them will happen to work, the other > will fail badly. > > Why is it that Python interprets them this way, and doesn't even give > a warning? What happened to errors not passing silently? Or, looking > at this the other way: Is there a way to enable such warnings/errors? > I can't see one in 'python[3] -h', but if there's some way elsewhere, > that would be a useful thing to recommend to people (I already > recommend running Python 2 with -tt). > > ChrisA > I've long thought they should be errors, but in Python they're not even warnings. It's one thing to let a user be sloppy on a shell's commandline, but in a program, if you have an invalid escape sequence, it should be an invalid string literal, full stop. And Python doesn't even treat these invalid sequences the same (broken) way C does. The documentation explicitly says it's different than C. If you're going to be different, at least be strict. -- DaveA From rosuav at gmail.com Sun Feb 22 21:55:48 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 13:55:48 +1100 Subject: Unrecognized backslash escapes in string literals In-Reply-To: <85vbit73jd.fsf@benfinney.id.au> References: <85vbit73jd.fsf@benfinney.id.au> Message-ID: On Mon, Feb 23, 2015 at 1:41 PM, Ben Finney wrote: > Chris Angelico writes: > >> Why is it that Python interprets them this way, and doesn't even give >> a warning? > > Because the interpretation of those literals is unambiguous and correct. And it also implies that never, in the entire infinite future of Python development, will any additional escapes be invented - because then it'd be ambiguous (in versions up to X, "\s" means "\\s", and after that, "\s" means something else). > It's unfortunate that MS Windows inherited the incompatible ?backslash > is a path separator?, long after backslash was already established in > many programming languages as the escape character. I agree, the fault is primarily with Windows. But I've seen similar issues when people use /-\| for box drawing and framing and such; Windows paths are by far the most common case of this, but not the sole. >> Is there a way to enable such warnings/errors? > > A warning or error for a correctly formatted literal with an unambiguous > meaning would be an up-Pythonic thing to have. > ... > This has the advantage that it's the same escape character used for text > string literals in virtually every other programming language, so you're > not needing to learn anything unusual. And yet the treatment of the edge case differs. In C, for instance, you get a compiler warning, and then the backslash is removed and you're left with just the other character. The trouble isn't that people need to learn that backslashes are special in Python string literals. The trouble is that, especially when file names are frequently being written with uppercase first letters, it's very easy to have code that just so happens to work, without being reliable. Having spent some time working with paths like these: fn = "C:\Foo\Bar\Asdf.ext" and then to find that each of these fails, but in a different way: path = "C:\Foo\Bar\"; fn = path + "Asdf.ext" fn = "c:\foo\bar\asdf.ext" fn = "c:\users\myname\blah" would surely count as surprising. Particularly since the last one will work fine in Python 2 sans unicode_literals, and will then blow up in Python 3 - because, contrary to the "no additional escapes" assumption, Unicode strings introduced new escapes, which means that "\u0123" has different meaning in byte strings and Unicode strings. In fact, that's an exception to the usual rule of "upper case is safe", and it's one that *will* trip people up, thanks to the "C:\Users" directory on a modern Windows system. What's the betting people will blame the failure on Python 3 and/or Unicode, rather than on the sloppy use of escapes and the poor choice of path separator on a popular platform? ChrisA From davea at davea.name Sun Feb 22 22:01:52 2015 From: davea at davea.name (Dave Angel) Date: Sun, 22 Feb 2015 22:01:52 -0500 Subject: Unrecognized backslash escapes in string literals In-Reply-To: <85vbit73jd.fsf@benfinney.id.au> References: <85vbit73jd.fsf@benfinney.id.au> Message-ID: <54EA9820.908@davea.name> On 02/22/2015 09:41 PM, Ben Finney wrote: > Chris Angelico writes: > >> In Python, unrecognized escape sequences are treated literally, >> without (as far as I can tell) any sort of warning or anything. > > Right. Text strings literals are documented to work that way > , > which refers the reader to the language reference > . > >> Why is it that Python interprets them this way, and doesn't even give >> a warning? > > Because the interpretation of those literals is unambiguous and correct. Correct according to a misguided language definition. > > It's unfortunate that MS Windows inherited the incompatible ?backslash > is a path separator?, long after backslash was already established in > many programming languages as the escape character. Windows "inherited" it from DOS. But since Windows was nothing but a DOS shell for several years, that's not surprising. The historical problem came from CP/M's use of the forward slash for a switch-character. Since MSDOS/PCDOS/QDOS was trying to permit transliterated CP/M programs, and because subdirectories were an afterthought (version 2.0), they felt they needed to pick a different character. At one time, the switch-character could be set by the user, but most programs ignored that, so it died. > >> Is there a way to enable such warnings/errors? > > A warning or error for a correctly formatted literal with an unambiguous > meaning would be an up-Pythonic thing to have. > > I can see the motivation, but really the best solution is to learn that > the backslash is an escape character in Python text string literals. > > This has the advantage that it's the same escape character used for text > string literals in virtually every other programming language, so you're > not needing to learn anything unusual. > I might be able to buy that argument if it was done the same way, but as it says in: https://docs.python.org/3/reference/lexical_analysis.html#strings """Unlike Standard C, all unrecognized escape sequences are left in the string unchanged, i.e., the backslash is left in the result. (This behavior is useful when debugging: if an escape sequence is mistyped, the resulting output is more easily recognized as broken.) """ The word "broken" is an admission that this was a flawed approach. If it's broken, it should be an error. I'm not suggesting that the implementation should falsely trigger an error. But that the language definition should be changed to define it as an error. -- DaveA From davea at davea.name Sun Feb 22 22:07:14 2015 From: davea at davea.name (Dave Angel) Date: Sun, 22 Feb 2015 22:07:14 -0500 Subject: calling subprocess In-Reply-To: <54EA929F.70509@davea.name> References: <4e828264-5445-42fc-a647-c56f80fc86ab@googlegroups.com> <54EA929F.70509@davea.name> Message-ID: <54EA9962.9070808@davea.name> On 02/22/2015 09:38 PM, Dave Angel wrote: > On 02/22/2015 08:13 PM, jkuplinsky at gmail.com wrote: >> >> OK (1) sorry about for/from > > That's not what you should be sorry about. You should be sorry you > didn't use cut&paste. > >> (2) print() sounds nice, but fact is , no matter what I try, i always >> get C:\\apps instead of c:\apps. So in this sense print() doesn't help >> much. Obviously i'm doing something wrong -- which is what you perhaps >> call shotgun debugging; but that's why i'm asking. >> > > You probably are getting confused about the difference between str() and > repr(). If you print the repr() of a string, it'll add quotes around > it, and escape the unprintable codes. So it'll double the backslash. It > also turns a newline into \n, and tabs into \t, and so on. Very useful. > > That's also what happens when you print a list that contains strings. > The individual elements of the list are converted using repr(). Watch > for the quotes to get a strong hint about what you're seeing. > > If you don't get a positive handle on how string literals relate to > string variables, and on str() and repr(), and print(), you'll be > jumping around the problem instead of solving it. > Two other things I should have pointed out here. The debugger uses repr() to display things, when you have an unassigned expression. And you can solve a lot of problems by just using a forward slash for a directory separator. The forward slash is just as correct in most circumstances within a program. It's mainly on the command line that forward slash takes on a different meaning. > > > Back to your original problem, which had you trying to use > subprocess.call to change the current directory. Current directory is > effectively (or actually) depending on the OS involved) an environment > variable, and changes made in a child process are not magically returned > to the parent. > > But even though there is an os.chdir() in Python, you really shouldn't > use it. Long experience of many people show that you're better off > manipulating the directories you need explicitly, converting any > directory that's relative to something other than the current one, to an > absolute. > -- DaveA From ryan.stuart.85 at gmail.com Sun Feb 22 22:16:39 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Mon, 23 Feb 2015 03:16:39 +0000 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> Message-ID: On Mon Feb 23 2015 at 12:05:46 PM Paul Rubin wrote: > I don't see what the big deal is. I hear tons of horror stories about > threads and I believe them, but the thing is, they almost always revolve > around acquiring and releasing locks in the wrong order, forgetting to > lock things, stuff like that. So I've generally respected the terror > and avoided programming in that style, staying with a message passing > style that may take an efficiency hit but seems to avoid almost all > those problems. TM also helps with lock hazards and it's a beautiful > idea--I just haven't had to use it yet. The Python IRC channel seems to > rage against threads and promote Twisted for concurrency, but Twisted > has always reminded me of Java. I use threads in Python all the time > and haven't gotten bitten yet. > > Many people have written at length about why it's bad. The most recent example I have come across is here -> https://glyph.twistedmatrix.com/2014/02/unyielding.html It's not a specific Python problem. I must be in the limited crowd that believes that the GIL is a *feature* of Python. Then again, maybe it isn't so limited since the GIL-less python implementations haven't really taken off. I have yet to come across a scenario I couldn't solve with either Processes, NumPy or event loops. Yes, when using processes, the passing of data can be annoying sometimes. But that is far less annoying then trying to debug something that shares state across threads. It's great that you haven't been bitten yet. But, the evidence seems to suggest the either you *will* be bitten at some point or, you already have been, and you just don't know it. Cheers > > Writing multithreaded code is *hard*. It is not a programming model > > which comes naturally to most human beings. Very few programs are > > inherently parallelizable, although many programs have *parts* which > > can be successfully parallelized. > > Parallel algorithms are complicated and specialized but tons of problems > amount to "do the same thing with N different pieces of data", so-called > embarassingly parallel. The model is you have a bunch of worker threads > reading off a queue and processing the items concurrently. Sometimes > separate processes works equally well, other times it's good to have > some shared data in memory instead of communicating through sockets. If > the data is mutable then have one thread own it and access it only with > message passing, Erlang style. If it's immutable after initialization > (protect it with a condition variable til initialization finishes) then > you can have read-only access from anywhere. > > > if you're running single-thread code on a single-core machine and > > still complaining about the GIL, you have no clue. > > Even the Raspberry Pi has 4 cores now, and fancy smartphones have had > them for years. Single core cpu's are specialized and/or historical. > > > for some programs, the simplest way to speed it up is to vectorize the > > core parts of your code by using numpy. No threads needed. > > Nice for numerical codes, not so hot for anything else. > > > Where are the people flocking to use Jython and IronPython? > > Shrug, who knows, those implementations were pretty deficient from what > heard. > > > For removal of the GIL to really make a difference: ... > > - you must be performing a task which is parallelizable and not > inherently > > sequential (no point using multiple threads if each thread spends all its > > time waiting for the previous thread); > > That's most things involving concurrency these days. > > > - the task must be one that moving to some other multi-processing model > > (such as greenlets, multiprocess, etc.) is infeasible; > > I don't understand this--there can be multiple ways to solve a problem. > > > - your threading bottleneck must be primarily CPU-bound, not I/O bound > > (CPython's threads are already very effective at parallelising I/O > tasks); > > If your concurrent program's workload makes it cpu-bound even 1% of the > time, then you gain something by having it use your extra cores at those > moments, instead of having those cores always do nothing. > > > - and you must be using libraries and tools which prevent you moving to > > Jython or IronPython or some other alternative. > > I don't get this at all. Why should I not want Python to have the same > capabilities? > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sun Feb 22 22:20:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 14:20:38 +1100 Subject: Unrecognized backslash escapes in string literals In-Reply-To: <85vbit73jd.fsf@benfinney.id.au> References: <85vbit73jd.fsf@benfinney.id.au> Message-ID: On Mon, Feb 23, 2015 at 1:41 PM, Ben Finney wrote: > Right. Text strings literals are documented to work that way > , > which refers the reader to the language reference > . BTW, quoting from that: """ Unlike Standard C, all unrecognized escape sequences are left in the string unchanged, i.e., the backslash is left in the result. (This behavior is useful when debugging: if an escape sequence is mistyped, the resulting output is more easily recognized as broken.) """ I'm not sure it's more obviously broken. Comparing Python and Pike: >>> "asdf\qwer" 'asdf\\qwer' > "asdf\qwer"; (1) Result: "asdfqwer" Which is the "more easily recognized as broken" depends on what the actual intention was. If you wanted to have a backslash (eg a path name), then the second one is, because you've just run two path components together. If you wanted to have some sort of special character ("\n"), then they're both going to be about the same - you'd expect to see "\n" in the output, one has added a backslash (assuming you're looking at the repr), the other has removed it. Likewise if you wanted some other symbol (eg forward slash), they're about the same (a doubled backslash, or a complete omission, same diff). But if you just fat-fingered a backslash into a string where it completely doesn't belong, then seeing a doubled backslash is definitely better than seeing just the following character (which would mask the error entirely). Since the interpreter can't know what the intention was, it obviously has to do just one thing and stick with it. I'm not convinced this is really an advantage. Python has been aiming more and more towards showing problems immediately, rather than having them depend on your data - for instance, instead of letting you treat bytes and characters as identical until you hit something that isn't ASCII, Py3 forces you to distinguish from the start. That said, though, there's probably a lot of code out there that depends on backslashes being non-special, so it's quite probably something that can't be changed. But it'd be nice to be able to turn on a warning for it. ChrisA From rosuav at gmail.com Sun Feb 22 22:25:25 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 14:25:25 +1100 Subject: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> Message-ID: On Mon, Feb 23, 2015 at 2:16 PM, Ryan Stuart wrote: > Many people have written at length about why it's bad. The most recent > example I have come across is here -> > https://glyph.twistedmatrix.com/2014/02/unyielding.html > > It's not a specific Python problem. I must be in the limited crowd that > believes that the GIL is a *feature* of Python. Then again, maybe it isn't > so limited since the GIL-less python implementations haven't really taken > off. The GIL isn't a problem, per se. It's a solution to an underlying problem (concurrent access to internal data structures) which comes with its own tradeoffs. Every method of eliminating the GIL is really an alternate solution to the same underlying problem, with its own tradeoffs. The GIL has simplicity on its side. ChrisA From ben+python at benfinney.id.au Sun Feb 22 22:35:25 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 23 Feb 2015 14:35:25 +1100 Subject: Unrecognized backslash escapes in string literals References: <85vbit73jd.fsf@benfinney.id.au> Message-ID: <85r3th711e.fsf@benfinney.id.au> Chris Angelico writes: > That said, though, there's probably a lot of code out there that > depends on backslashes being non-special, so it's quite probably > something that can't be changed. But it'd be nice to be able to turn > on a warning for it. If you're motivated to see such warnings, an appropriate place to implement them would be in PyLint or another established static code analysis tool. -- \ ?The whole area of [treating source code as intellectual | `\ property] is almost assuring a customer that you are not going | _o__) to do any innovation in the future.? ?Gary Barnett | Ben Finney From no.email at nospam.invalid Sun Feb 22 22:45:45 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 19:45:45 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> Message-ID: <87lhjpb89i.fsf@jester.gateway.pace.com> Ryan Stuart writes: > Many people have written at length about why it's bad. The most recent > example I have come across is here -> > https://glyph.twistedmatrix.com/2014/02/unyielding.html That article is about the hazards of mutable state shared between threads. The key to using threads safely is to not do that. So the "transfer" example in the article would instead be a message handler in the thread holding the account data, and it would do the transfer in the usual sequential way. You'd start a transfer by sending a message through a Queue, and get back a reply through another queue. In Erlang that style is enforced: it has basically no such thing as data mutation or sharing. In Python it's straightforward to write in that style; it's just that the language doesn't stop you from departing from it, sort of like a language with weak type-checking. You can survive it if the code complexity isn't too bad. In most programs I've dealt with, the number of distinct handlers is not all that large (there might be 1000 threads in a high-concurrency network server, but they're all doing about the same thing). So it hasn't been that hard to "color inside the lines" with a bit of thoughtfulness and code inspection. You might like this: http://jlouisramblings.blogspot.com/2012/08/getting-25-megalines-of-code-to-behave.html From ryan.stuart.85 at gmail.com Sun Feb 22 23:00:16 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Mon, 23 Feb 2015 04:00:16 +0000 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> Message-ID: On Mon Feb 23 2015 at 1:50:40 PM Paul Rubin wrote: > That article is about the hazards of mutable state shared between > threads. The key to using threads safely is to not do that. So the > "transfer" example in the article would instead be a message handler in > the thread holding the account data, and it would do the transfer in the > usual sequential way. You'd start a transfer by sending a message > through a Queue, and get back a reply through another queue. > I think that is a pretty accurate summary. In fact, the article even says that. So, just to iterate its point, if you are using non-blocking Queues to communicate to these threads, then you just have a communicating event loop. Given that Queues work perfectly with with processes as well, what is the point of using a thread? Using a process/fork is far safer in that someone can't "accidentally" decide to alter mutable state in the future. > You might like this: > > http://jlouisramblings.blogspot.com/2012/08/getting- > 25-megalines-of-code-to-behave.html Thanks for this, I'll take a look. Cheers > > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Feb 23 01:02:13 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Feb 2015 01:02:13 -0500 Subject: id() and is operator In-Reply-To: <871tlhiqpz.fsf@elektro.pacujo.net> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> <871tlhiqpz.fsf@elektro.pacujo.net> Message-ID: On 2/22/2015 4:25 PM, Marko Rauhamaa wrote: > LJ : > >>>>> id(b[0]) >> 45855552 > [...] >>>>> id(b[2]) >> 45855552 >> Please correct me if I am wrong, but according to this b[2] and b[0] >> are the same object. Now, >> >>>>> b[0] is b[2] >> False > > This is a true statement: > > If X is Y, then id(X) == id(Y). > > However, this is generally not a true statement: > > If X is Y, then id(X) is id(Y). If X and Y exist at the *same time*, then (X is Y) == (id(X) is id(Y)). Since X and Y in the example above do not exist at the same time, it is nonsensical to compare them. -- Terry Jan Reedy From no.email at nospam.invalid Mon Feb 23 01:13:49 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 22 Feb 2015 22:13:49 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> Message-ID: <87h9udb1eq.fsf@jester.gateway.pace.com> Ryan Stuart writes: > I think that is a pretty accurate summary. In fact, the article even > says that. So, just to iterate its point, if you are using > non-blocking Queues to communicate to these threads, then you just > have a communicating event loop. Given that Queues work perfectly with > with processes as well, what is the point of using a thread? What do you mean about Queues working with processes? I meant Queue.Queue. There is multiprocessing.Queue but that's much less capable, and it uses cumbersome IPC like pipes or sockets instead of a lighter weight lock. Threads can also share read-only data and you can pass arbitrary objects (such as code callables that you want the other thread to execute--this is quite useful) through Queue.Queue. I don't think you can do that with the multiprocessing module. From tjreedy at udel.edu Mon Feb 23 01:14:03 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Feb 2015 01:14:03 -0500 Subject: id() and is operator In-Reply-To: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> Message-ID: On 2/22/2015 12:53 PM, LJ wrote: > Hi everyone. Quick question here. Lets suppose if have the > following numpy array: > b=np.array([[0]*2]*3) > > and then: > >>>> id(b[0]) > 45855552 >>>> id(b[1]) > 45857512 >>>> id(b[2]) > 45855552 > > Please correct me if I am wrong, You are, as other explained > but according to this b[2] and b[0] are the same object. >>>> b[0] is b[2] > False > > Any clarification is much appreciated. In Python, 'two' objects can only be the same thing if they exist simultaneously. Retry your experiment with simultaneous objects. >>> b0 = b[0] >>> b1 = b[1] >>> b2 = b[2] # etc. The three objects will have different ids. The mail purpose of the id function is to text properties of a particular implementation. It also has uses with ctypes. Beginners should generally ignore it, and certainly not touch it until reading and understanding its doc. The main use for 'is' is for 'is None' comparison. -- Terry Jan Reedy From gherron at digipen.edu Mon Feb 23 01:29:50 2015 From: gherron at digipen.edu (Gary Herron) Date: Sun, 22 Feb 2015 22:29:50 -0800 Subject: id() and is operator In-Reply-To: References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> <871tlhiqpz.fsf@elektro.pacujo.net> Message-ID: <54EAC8DE.60702@digipen.edu> On 02/22/2015 10:02 PM, Terry Reedy wrote: > On 2/22/2015 4:25 PM, Marko Rauhamaa wrote: >> LJ : >> >>>>>> id(b[0]) >>> 45855552 >> [...] >>>>>> id(b[2]) >>> 45855552 > >>> Please correct me if I am wrong, but according to this b[2] and b[0] >>> are the same object. Now, >>> >>>>>> b[0] is b[2] >>> False >> >> This is a true statement: >> >> If X is Y, then id(X) == id(Y). >> >> However, this is generally not a true statement: >> >> If X is Y, then id(X) is id(Y). > > If X and Y exist at the *same time*, then (X is Y) == (id(X) is > id(Y)). Since X and Y in the example above do not exist at the same > time, it is nonsensical to compare them. Not quite. You've been bitten by the "is" versus "==" trap. You could use id(X)==id(Y) but not id(X) is id(Y) not even if X and Y are the same object. Simple examples: >>> a=3 >>> id(a) is id(a) False >>> a=3 >>> b=a >>> id(a) is id(b) False The explanation is that each call to id() makes its own independent Python integer object containing the large integer (10771264 in this case). The two integer objects satisfy "==", but they are separate Python objects so they do not satisfy "is". As a side note, It is an implementation detail whether two Python integer objects created independently but with the same value are separate objects or references to a single object. CPython caches small integers so that only one integer object of each value exists, but not so for large integers. You can experiment with the cutoff on your particular flavor of Python. On mine (Python 3.4.2 (default, Oct 8 2014, 13:08:17) ;[GCC 4.9.1] on linux) it's somewhere between 200 and 300: >>> 201 is 1+200 True >>> 301 is 1+300 False Gary Herron -- Dr. Gary Herron Department of Computer Science DigiPen Institute of Technology (425) 895-4418 From marko at pacujo.net Mon Feb 23 01:31:04 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 08:31:04 +0200 Subject: id() and is operator References: <87f18c68-120d-44f2-bd34-6f73c69365da@googlegroups.com> <871tlhiqpz.fsf@elektro.pacujo.net> Message-ID: <87k2z9gmvr.fsf@elektro.pacujo.net> Terry Reedy : > On 2/22/2015 4:25 PM, Marko Rauhamaa wrote: >> This is a true statement: >> >> If X is Y, then id(X) == id(Y). >> >> However, this is generally not a true statement: >> >> If X is Y, then id(X) is id(Y). > > If X and Y exist at the *same time*, then (X is Y) == (id(X) is id(Y)). Sorry, you're wrong: Python 3.3.2 (default, Dec 4 2014, 12:49:00) [GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> x = "alksjdlfkajsf" >>> id(x) is id(x) False >>> (x is x) == (id(x) is id(x)) False Marko From tjreedy at udel.edu Mon Feb 23 01:34:19 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Feb 2015 01:34:19 -0500 Subject: Future of Pypy? In-Reply-To: References: Message-ID: On 2/22/2015 7:45 AM, Dave Farrance wrote: > As an engineer, I can quickly knock together behavioural models of > electronic circuits, complete units, and control systems in Python, then > annoyingly in a few recent cases, have to re-write in C for speed. > > I've tried PyPy, the just-in-time compiler for Python, and that is > impressively, hugely fast in comparison, but it's no good making these > models if I can't display the results in a useful way, and at the moment > PyPy just doesn't have the huge range of useful time-saving libraries that > CPython has. It's still quicker to do a re-write in the more cumbersome C > than try to work with PyPy because C, like CPython, also has many useful > libraries. > > A few years back, I recall people saying that PyPy was going to be the > future of Python, but it seems to me that CPython still has the lion's > share of the momentum, is developing faster and has ever more libraries, > while PyPy is struggling to get enough workers to even get Numpy > completed. > > Maybe there's not enough people like me that have really felt the need for > the speed. Or maybe it's simply the accident of the historical > development path that's set-in-stone an interpreter rather than a JIT. > Anybody got a useful perspective on this? > -- Terry Jan Reedy From ian.g.kelly at gmail.com Mon Feb 23 01:57:04 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 22 Feb 2015 23:57:04 -0700 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 22, 2015 at 7:22 AM, Cem Karan wrote: > > On Feb 22, 2015, at 5:15 AM, Gregory Ewing wrote: > >> Frank Millman wrote: >>> "In order to inform users that certain bits of state have changed, I require them to register a callback with my code." >>> This sounds to me like a pub/sub scenario. When a 'listener' object comes into existence it is passed a reference to a 'controller' object that holds state. It wants to be informed when the state changes, so it registers a callback function with the controller. >> >> Perhaps instead of registering a callback function, you >> should be registering the listener object together with >> a method name. >> >> You can then keep a weak reference to the listener object, >> since if it is no longer referenced elsewhere, it presumably >> no longer needs to be notified of anything. > > I see what you're saying, but I don't think it gains us too much. If I store an object and an unbound method of the object, or if I store the bound method directly, I suspect it will yield approximately the same results. Well, it ties the weak ref to the lifetime of the object owning the callback rather than to the lifetime of the potentially unreferenced callback itself. I'm not fond of the scheme though because it forces the callback to be a method, and I'd prefer not to make that assumption. Also, I just noticed that Python 3.4 adds a weakref.WeakMethod class that solves the problem for the bound method case. That still leaves the closure and lambda cases, but here's a thought: add an optional argument to the callback registration method that specifies what object to tie the weak ref to. Something like: class Listenable: def __init__(self): self._callbacks = weakref.WeakKeyDictionary() def listen(self, callback, owner=None): if owner is None: if isinstance(callback, types.MethodType): owner = weakref.WeakMethod(callback) else: owner = callback self._callbacks.setdefault(owner, []).append(callback) def do_callbacks(self, message): for callbacks in self._callbacks.values(): for callback in callbacks: callback(message) From ryan.stuart.85 at gmail.com Mon Feb 23 02:32:00 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Mon, 23 Feb 2015 07:32:00 +0000 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> Message-ID: On Mon Feb 23 2015 at 4:15:42 PM Paul Rubin wrote: > > What do you mean about Queues working with processes? I meant > Queue.Queue. There is multiprocessing.Queue but that's much less > capable, and it uses cumbersome IPC like pipes or sockets instead of a > lighter weight lock. Threads can also share read-only data and you can > pass arbitrary objects (such as code callables that you want the other > thread to execute--this is quite useful) through Queue.Queue. I don't > think you can do that with the multiprocessing module. > These things might be convenient but they are error prone for the reasons pointed out. Also, the majority can be achieved via the process approach. For example, using fork to take a copy of the current process (including the heap) you want to use will give you access to any callables on the heap. The vital point here is that fork takes a *copy* of the process and runs in a *separate* memory space. This means there can be no accidents here. If it were to run in the same memory space like a thread then bugs anywhere in the code you run could cause very nasty problems. This includes not only bugs in your code, but also bugs in any other library code. Not only are they nasty, they could be close to invisible. And when I saw any code you run, I literally mean any. Even if you are extra careful to not touch any shared state in your code, you can almost be guaranteed that code higher up the stack, like malloc for example, *will* be using shared state. The risk of unintended and difficult to track issues when using threads is very high because of shared state. Even if you aren't sharing state in your code directly, code higher up the stack will be sharing state. That is the whole point of a thread, that's what they were invented for. Using threads safely might well be impossible much less verifiable. So when there are other options that are just as viable/functional, result in far less risk and are often much quicker to implement correctly, why wouldn't you use them? If it were easy to use threads in a verifiably safe manner, then there probably wouldn't be a GIL. Cheers > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Mon Feb 23 02:41:29 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 23 Feb 2015 18:41:29 +1100 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> Message-ID: <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Paul Rubin wrote: > Steven D'Aprano writes: >> I'm sorry, but the instant somebody says "eliminate the GIL", they lose >> credibility with me. Yes yes, I know that in *your* specific case you've >> done your research and (1) multi-threaded code is the best solution for >> your application and (2) alternatives aren't suitable. > > I don't see what the big deal is. I hear tons of horror stories about > threads and I believe them, but the thing is, they almost always revolve > around acquiring and releasing locks in the wrong order, forgetting to > lock things, stuff like that. Deadlocks, livelocks, and the simple fact that debugging threaded code is much, much, much harder than debugging single-thread code. You'll notice that nowhere in my post did I say "Don't use threads!". But they aren't a panacea, and there are other concurrency models available which don't have the same disadvantages as threads (they have their own pros and cons instead). [...] >> For removal of the GIL to really make a difference: ... [...] >> - the task must be one that moving to some other multi-processing model >> (such as greenlets, multiprocess, etc.) is infeasible; > > I don't understand this--there can be multiple ways to solve a problem. Yes, but my point is that if some other way solves the problem, then you should *use that other technique* rather than complain about the GIL. The GIL is not a bottleneck if you can bypass it. I can sympathize with somebody who says "I have this problem that can only be solved with threads, anything else is too difficult, but the GIL makes it too slow". But I don't sympathize with somebody who says "I have this problem that can be solved with threads or multiprocessing, but I insist that only threads can be used. And now the GIL makes it too slow. Python sux." No. You [generic you] sux, for not investigating whether multiprocessing will do the job. It even has the same public API as threads. >> - your threading bottleneck must be primarily CPU-bound, not I/O bound >> (CPython's threads are already very effective at parallelising I/O >> tasks); > > If your concurrent program's workload makes it cpu-bound even 1% of the > time, then you gain something by having it use your extra cores at those > moments, instead of having those cores always do nothing. Not necessarily. Threading has overhead too. Even in GIL-less Python, two threads aren't twice as fast as one thread. So there comes a point of diminishing returns, where the benefit you get from threading is less than the cost of threading. Now your code will actually run slower with threads, GIL or no GIL. >> - and you must be using libraries and tools which prevent you moving to >> Jython or IronPython or some other alternative. > > I don't get this at all. Why should I not want Python to have the same > capabilities? Python does have the same capabilities. Jython and IronPython aren't different languages, they are Python. If you want *CPython* to work without a GIL, well, are you volunteering to do the work? It is a massive job, and the core devs aren't terribly interested. Probably because they understand that the GIL is not often an unsurmountable problem in real life. That brings us to the point I was making: I believe, and the core devs appear to think the same, that the *actual* number of people who would benefit from CPython losing the GIL is not large enough to justify the work it would require. -- Steve From marko at pacujo.net Mon Feb 23 03:16:01 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 10:16:01 +0200 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87zj85cabi.fsf@elektro.pacujo.net> Steven D'Aprano : > Yes, but my point is that if some other way solves the problem, then > you should *use that other technique* rather than complain about the > GIL. The GIL is not a bottleneck if you can bypass it. > > I can sympathize with somebody who says "I have this problem that can > only be solved with threads, anything else is too difficult, but the > GIL makes it too slow". > > But I don't sympathize with somebody who says "I have this problem > that can be solved with threads or multiprocessing, but I insist that > only threads can be used. And now the GIL makes it too slow. Python > sux." I'm as "antithread" as anybody, but I think you are blaming the victim here. I do sympathize with a person who wants to use threads and finds out there is no true parallelism. GIL is a downside you have to accept when using a language like Python that has to "hold the world still" while it's manipulating its data structures in a safe way. Because of the dynamism involved, effective parallelism is difficult to achieve. I must say I have never been bitten by this problem because I only use threads under duress (occasionally when dealing with blocking APIs) and because I have very modest performance requirements for Python. For anything that needs to be fast, I use other programming languages. Currently, my work is maybe 45% bash, 45% Python and 10% C/C++ (and even in my C code, I generally steer clear of threads). Marko From __peter__ at web.de Mon Feb 23 03:41:42 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 23 Feb 2015 09:41:42 +0100 Subject: Unrecognized backslash escapes in string literals References: <85vbit73jd.fsf@benfinney.id.au> <85r3th711e.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Chris Angelico writes: > >> That said, though, there's probably a lot of code out there that >> depends on backslashes being non-special, so it's quite probably >> something that can't be changed. But it'd be nice to be able to turn >> on a warning for it. > > If you're motivated to see such warnings, an appropriate place to > implement them would be in PyLint or another established static code > analysis tool. Pylint already produces a warning. However, it cannot read the author's mind: $ cat tmp.py print("C:\alpha") print("C:\beta") print("C:\gamma") $ pylint tmp.py ************* Module tmp W: 3, 0: Anomalous backslash in string: '\g'. String constant might be missing an r prefix. (anomalous-backslash-in-string) C: 1, 0: Missing module docstring (missing-docstring) The same would go for a warning built into the compiler. Maybe having editors highlight the special combinations would be the more helpful approach. A tooltip could explain the meaning. From rosuav at gmail.com Mon Feb 23 04:19:34 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 20:19:34 +1100 Subject: Future of Pypy? In-Reply-To: <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 23, 2015 at 6:41 PM, Steven D'Aprano wrote: >>> - and you must be using libraries and tools which prevent you moving to >>> Jython or IronPython or some other alternative. >> >> I don't get this at all. Why should I not want Python to have the same >> capabilities? > > Python does have the same capabilities. Jython and IronPython aren't > different languages, they are Python. Presuming that he meant CPython, one other good reason for wanting to stick with it rather than jump to another interpreter is all the new features of the recent versions. (Do Jython or IronPython support Python 3.x at all, even?) Every new release of CPython has new and cool features, and the other Pythons are generally playing catch-up. ChrisA From pfranken85 at gmail.com Mon Feb 23 04:48:15 2015 From: pfranken85 at gmail.com (pfranken85 at gmail.com) Date: Mon, 23 Feb 2015 01:48:15 -0800 (PST) Subject: Question on asyncio In-Reply-To: <8761atiqvt.fsf@elektro.pacujo.net> References: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> <8761atiqvt.fsf@elektro.pacujo.net> Message-ID: <5c9fa6a5-f92a-42a8-a760-9477f9d8d0d2@googlegroups.com> Hello Marko! Am Sonntag, 22. Februar 2015 22:21:55 UTC+1 schrieb Marko Rauhamaa: > In asyncio, you typically ignore the value returned by yield. While > generators use yield to communicate results to the calling program, > coroutines use yield only as a "trick" to implement cooperative > multitasking and an illusion of multithreading. Really? I saw several exmaples, where a coroutine returned a value which was then picked up from the yield from statement... > > Thus, "yield from" in asyncio should be read, "this is a blocking > state." > > > Is this the way one would accomplish this task? Or are there better > > ways? Should read_value_from_device() be a @coroutine as well? It may > > contain parts that take a while ... Of course, instead of print(new) I > > would add the corresponding calls for notifying the client about the > > update. > > How do you read a value from the hardware? Do you use a C extension? Do > you want read_value_from_device() to block until the hardware has the > value available or is the value always available for instantaneous > reading? > > If the value is available instantaneously, you don't need to turn it > into a coroutine. However, if blocking is involved, you definitely > should do that. Depending on your hardware API it can be easy or > difficult. If you are running CPython over linux, hardware access > probably is abstracted over a file descriptor and a coroutine interface > would be simple. The corresponding call is a call to the python smbus library. It includes several sleeps (even though they are only about 50ms). Therefore I think it is worthwhile to encapsulate it into a coroutine. However I am not quite sure, how I should call it and integrate it into the main loop. In particular, do I need something like the runner routine() with and infinite while loop or can just add the check() routine to the main loop, so that it gets executed regularly and call the corresponding notifier whenever the condition of the if statement is true? Thanks for your input! From storchaka at gmail.com Mon Feb 23 05:14:08 2015 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 23 Feb 2015 12:14:08 +0200 Subject: Unrecognized backslash escapes in string literals In-Reply-To: References: <85vbit73jd.fsf@benfinney.id.au> Message-ID: On 23.02.15 04:55, Chris Angelico wrote: > I agree, the fault is primarily with Windows. But I've seen similar > issues when people use /-\| for box drawing and framing and such; > Windows paths are by far the most common case of this, but not the > sole. There is also issues with regular expressions. >>> import re >>> re.match(r'a\s', 'a ') <_sre.SRE_Match object; span=(0, 2), match='a '> >>> re.match('a\s', 'a ') <_sre.SRE_Match object; span=(0, 2), match='a '> >>> re.match(r'a\b', 'a ') <_sre.SRE_Match object; span=(0, 1), match='a'> >>> re.match('a\b', 'a ') Oops. 'a\s' works the same as r'a\s', but 'a\b' works different from r'a\b'. From davecook at nowhere.net Mon Feb 23 06:36:47 2015 From: davecook at nowhere.net (Dave Cook) Date: 23 Feb 2015 11:36:47 GMT Subject: Future of Pypy? References: Message-ID: <54eb10cf$0$44508$c3e8da3$66d3cc2f@news.astraweb.com> On 2015-02-22, Dave Farrance wrote: > It's still quicker to do a re-write in the more cumbersome C You should try Cython. Dave From cfkaran2 at gmail.com Mon Feb 23 06:47:10 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Mon, 23 Feb 2015 06:47:10 -0500 Subject: Design thought for callbacks In-Reply-To: <201502222229.t1MMT7Dn029607@fido.openend.se> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net><0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> <201502222229.t1MMT7Dn029607@fido.openend.se> Message-ID: On Feb 22, 2015, at 5:29 PM, Laura Creighton wrote: > In a message of Sun, 22 Feb 2015 17:09:01 -0500, Cem Karan writes: > >> Documentation is a given; it MUST be there. That said, documenting >> something, but still making it surprising, is a bad idea. For >> example, several people have been strongly against using a WeakSet to >> hold callbacks because they expect a library to hold onto callbacks. >> If I chose not to do that, and used a WeakSet, then even if I >> documented it, it would still end up surprising people (and from the >> sound of it, more people would be surprised than not). > >> Thanks, Cem Karan > > No matter what you do, alas, will surprise the hell out of people > because callbacks do not behave as people expect. Among people who > have used callbacks, what you are polling is 'what are people > familiar with', and it seems for the people around here, now, > WeakSets are not what they are familiar with. And that's fine. I know that regardless of what I do, some people are going to be surprised. I'm trying to develop APIs that reduce that surprise as far as possible. That means I can spend more time coding and less time answering questions... :) > But that is not so surprising. How many people use WeakSets for > _anything_? I've never used them, aside from 'ooh! cool shiny > new language feature! Let's kick it around the park!' That people > aren't familiar with WeakSets doesn't mean all that much. Actually, I use them when building caches of stuff, and I use weak references when I have trees of stuff so the child nodes know of, but don't hold onto, their parents. But I agree with you, there aren't a huge number of use-cases. > The question I have is does this architecture make things harder, > easier or about the same to debug? To write tests for? to do Test > Driven Design with? Good questions! That was why I was asking about 'gotchas' with WeakSets originally. Honestly, the only way to know for sure would be to write two APIs for doing similar things, and then see how people react to them. The problem is, how do you set up such a study so it is statistically valid? Cem From cloverobert at gmail.com Mon Feb 23 07:13:46 2015 From: cloverobert at gmail.com (Robert Clove) Date: Mon, 23 Feb 2015 19:13:46 +0700 Subject: bufsize must be an integer in subprocess.Popen Message-ID: Hi All, I am using the Linux system with python, i am running the following script #!/usr/bin/python import threading import time import sys import subprocess import datetime import os import time import logging proc1=subprocess.Popen("/root/Desktop/abc.py","64","abc",shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE) In this script i am calling the other script named abc.py which is located on the desktop with 64 and abc as its arguments I am getting the following error File "/usr/lib64/python2.6/subprocess.py", line 589, in __init__ raise TypeError("bufsize must be an integer") TypeError: bufsize must be an integer Can you tell me why and whats the sol? From songofacandy at gmail.com Mon Feb 23 07:17:57 2015 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 23 Feb 2015 21:17:57 +0900 Subject: bufsize must be an integer in subprocess.Popen In-Reply-To: References: Message-ID: When `shell=True`, the first argument should be string passed to shell. So you should: proc1=subprocess.Popen("/root/Desktop/abc.py 64 abc", shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE) On Mon, Feb 23, 2015 at 9:13 PM, Robert Clove wrote: > Hi All, > > I am using the Linux system with python, i am running the following script > > #!/usr/bin/python > > > > import threading > > import time > > import sys > import subprocess > import datetime > import os > import time > import logging > > > > proc1=subprocess.Popen("/root/Desktop/abc.py","64","abc",shell=True,stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > > > In this script i am calling the other script named abc.py which is > located on the desktop with 64 and abc as its arguments > I am getting the following error > File "/usr/lib64/python2.6/subprocess.py", line 589, in __init__ > raise TypeError("bufsize must be an integer") > TypeError: bufsize must be an integer > > > Can you tell me why and whats the sol? > -- > https://mail.python.org/mailman/listinfo/python-list -- INADA Naoki From shgck at pistache.land Mon Feb 23 07:22:18 2015 From: shgck at pistache.land (Shgck) Date: Mon, 23 Feb 2015 13:22:18 +0100 Subject: bufsize must be an integer in subprocess.Popen In-Reply-To: References: Message-ID: <54EB1B7A.2000701@pistache.land> On 23/02/2015 13:13, Robert Clove wrote: > Hi All, > > I am using the Linux system with python, i am running the following script > > #!/usr/bin/python > > > > import threading > > import time > > import sys > import subprocess > import datetime > import os > import time > import logging > > > > proc1=subprocess.Popen("/root/Desktop/abc.py","64","abc",shell=True,stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > > > In this script i am calling the other script named abc.py which is > located on the desktop with 64 and abc as its arguments > I am getting the following error > File "/usr/lib64/python2.6/subprocess.py", line 589, in __init__ > raise TypeError("bufsize must be an integer") > TypeError: bufsize must be an integer > > > Can you tell me why and whats the sol? Hi, the parameter list should be a list of strings, not several unpacked strings : command = ["/root/Desktop/abc.py","64","abc"] proc1 = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) Also, be sure to use a shebang like "#!/usr/bin/env python" at the beginning of abc.py or your environment may be unable to find out that you want to execute that script with Python. Or better, you can call that script directly with Python : command = ["python","/root/Desktop/abc.py","64","abc"] From frank at chagford.com Mon Feb 23 07:29:14 2015 From: frank at chagford.com (Frank Millman) Date: Mon, 23 Feb 2015 14:29:14 +0200 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net><0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> <201502222229.t1MMT7Dn029607@fido.openend.se> Message-ID: "Cem Karan" wrote in message news:A3C11A70-5846-4915-BB26-B23793B65670 at gmail.com... > > > Good questions! That was why I was asking about 'gotchas' with WeakSets > originally. Honestly, the only way to know for sure would be to write two > APIs for doing similar things, and then see how people react to them. The > problem is, how do you set up such a study so it is statistically valid? > Just in case you missed Steven's comment on my 'gotcha', and my reply, it is worth repeating that what I reported as a gotcha was not what it seemed. If you set up the callback as a weakref, and the listening object goes out of scope, it will wait to be garbage collected. However, as far as I can tell, the weakref is removed at the same time as the object is gc'd, so there is no 'window' where the weakref exists but the object it is referencing does not exist. My problem was that I had performed a cleanup operation on the listening object before letting it go out of scope, and it was no longer in a valid state to deal with the callback, resulting in an error. If you do not have that situation, your original idea may well work. Frank From songofacandy at gmail.com Mon Feb 23 07:33:19 2015 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 23 Feb 2015 21:33:19 +0900 Subject: bufsize must be an integer in subprocess.Popen In-Reply-To: <54EB1B7A.2000701@pistache.land> References: <54EB1B7A.2000701@pistache.land> Message-ID: > Hi, the parameter list should be a list of strings, not several unpacked > strings : > > command = ["/root/Desktop/abc.py","64","abc"] > proc1 = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > The first argument is list of string only when `shell=False` (default). When `shell=True`, you should one string as the argument. -- INADA Naoki From rosuav at gmail.com Mon Feb 23 07:42:41 2015 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Feb 2015 23:42:41 +1100 Subject: bufsize must be an integer in subprocess.Popen In-Reply-To: References: Message-ID: On Mon, Feb 23, 2015 at 11:13 PM, Robert Clove wrote: > proc1=subprocess.Popen("/root/Desktop/abc.py","64","abc",shell=True,stdout=subprocess.PIPE, > stderr=subprocess.PIPE) As others have said, you need to use either a single command or a list of strings. But why are you using shell=True here? You appear to be invoking one single script, and not needing shell features at all. Additionally, it seems highly likely that you're running this as root (the /root directory is usually the root user's home dir, and is readable by nobody else), so any sort of shell injection exploit could easily result in an attacker getting full root access to your box. I strongly suggest using shell=False here. ChrisA From nomail at invalid.com Mon Feb 23 07:55:47 2015 From: nomail at invalid.com (ast) Date: Mon, 23 Feb 2015 13:55:47 +0100 Subject: list storing variables Message-ID: <54eb2357$0$3011$426a74cc@news.free.fr> hi >>> a = 2; b = 5 >>> Li = [a, b] >>> >>> Li [2, 5] >>> a=3 >>> Li [2, 5] >>> Ok, a change in a or b doesn't impact Li. This works as expected Is there a way to define a container object able to store some variables so that a change of a variable make a change in this object content ? I dont need this feature. It is just something I am thinking about. In C language, there is &A for address of A From davea at davea.name Mon Feb 23 08:24:02 2015 From: davea at davea.name (Dave Angel) Date: Mon, 23 Feb 2015 08:24:02 -0500 Subject: list storing variables In-Reply-To: <54eb2357$0$3011$426a74cc@news.free.fr> References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: <54EB29F2.2080202@davea.name> On 02/23/2015 07:55 AM, ast wrote: > hi > >>>> a = 2; b = 5 >>>> Li = [a, b] >>>> >>>> Li > [2, 5] >>>> a=3 >>>> Li > [2, 5] >>>> > > Ok, a change in a or b doesn't impact Li. This works as expected > > Is there a way to define a container object able to store some variables > so that a change of a variable make a change in this object content ? > > I dont need this feature. It is just something I am thinking about. > > In C language, there is &A for address of A > When you do an "a=3" you are rebinding a, and that has no connection to what's in the list. If you were to modify the object that a and L1[0] share, then yes, the modifications would affect both sides. However, an int is immutable, so you cannot directly do it. The simplest approximation to what you're asking is to use a list within the list. a = [42]; b = 65 L1 = [a, b] a[0] = 99 #this doesn't rebind a, just changes the list # object it is bound to print(L1) yields: [[99], 65] Clearly, instead of a list, you could use some other mutable object. In fact, you could use something like: class Dummy(object): pass a = Dummy() a.value = 12 ... -- DaveA From marko at pacujo.net Mon Feb 23 08:27:47 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 15:27:47 +0200 Subject: Question on asyncio References: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> <8761atiqvt.fsf@elektro.pacujo.net> <5c9fa6a5-f92a-42a8-a760-9477f9d8d0d2@googlegroups.com> Message-ID: <87k2z8dagc.fsf@elektro.pacujo.net> pfranken85 at gmail.com: > Hello Marko! > > Am Sonntag, 22. Februar 2015 22:21:55 UTC+1 schrieb Marko Rauhamaa: >> In asyncio, you typically ignore the value returned by yield. While >> generators use yield to communicate results to the calling program, >> coroutines use yield only as a "trick" to implement cooperative >> multitasking and an illusion of multithreading. > > Really? I saw several exmaples, where a coroutine returned a value > which was then picked up from the yield from statement... Actually, you might have better information in that regard than me. In my trials, I never used it like that. > The corresponding call is a call to the python smbus library. It > includes several sleeps (even though they are only about 50ms). > Therefore I think it is worthwhile to encapsulate it into a coroutine. Maybe. Then you'll probably have to rewrite smbus to work asyncio style. Those sleeps would then be done as "yield from" statements. Marko From steve+comp.lang.python at pearwood.info Mon Feb 23 08:31:05 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 00:31:05 +1100 Subject: Design thought for callbacks References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54eb2b9a$0$12986$c3e8da3$5496439d@news.astraweb.com> Cem Karan wrote: > > On Feb 21, 2015, at 12:27 PM, Steven D'Aprano > wrote: >> The simplest possible identity-based scheme would be something like this: >> >> >> # don't hate me for using a global variable >> CALLBACKS = [] >> >> def register(func): >> if func not in CALLBACKS: >> CALLBACKS.append(func) >> >> def unregister(func): >> try: >> CALLBACKS.remove(func) >> except ValueError: >> pass Oops! That's not identity-based, that's *equality* based. Both the `in` operator and the list `remove` method implicitly perform equality checks, not identity checks. Which means that they will work with methods, since method equality compares against the underlying function, which is the same: py> class Spam(object): ... def method(self): ... pass ... py> s = Spam() py> a = s.method py> b = s.method py> a is b False py> a == b True py> a.__func__ is b.__func__ True So, when I say this: >> That's probably a bit too simple, since it won't behave as expected with >> bound methods. The problem is that bound methods are generated on the >> fly, so this won't work: I was mistaken. > Are you sure about that? I just tested out the following code, and it > appears to work correctly: You are correct. -- Steven From jkn_gg at nicorp.f9.co.uk Mon Feb 23 08:34:28 2015 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Mon, 23 Feb 2015 05:34:28 -0800 (PST) Subject: Standard In-Reply-To: References: Message-ID: On Sunday, 22 February 2015 14:11:54 UTC, Mark Lawrence wrote: > On 19/02/2015 16:27, Phillip Fleming wrote: > > In my opinion, Python will not take off like C/C++ if there is no ANSI > > standard. > > > > Python has already taken off because it doesn't have a standard as such. > Bjarne Stroustrup, in 'The Design and Evolution of C++', says about the early days of C++ (or possibly, C with Classes), something to the effect that he knw that: "C++ had to be a weed, that you left it for a year and came back and found it in twenty new places, rather than a rose that needed tending" IMO Python has done very well being a similar kind of (very well behaved!) weed... Jon N From David.Aldrich at EMEA.NEC.COM Mon Feb 23 08:44:41 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Mon, 23 Feb 2015 13:44:41 +0000 Subject: Python shell: Arrow keys not working in PuTTY Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Hi I want to use the Python 3.4 interpreter interactively, via a PuTTY ssh session. Python is running on Centos 5. Currently, the arrow keys do not work: $ /usr/local/bin/python3.4 Python 3.4.2 (default, Feb 11 2015, 15:06:33) [GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ^[[A This stackoverflow thread: http://stackoverflow.com/questions/893053/python-shell-arrow-keys-do-not-work-on-remote-machine suggests that the problem can be fixed by installing the readline package: sudo apt-get install libreadline-dev followed by a rebuild of Python or pip install readline Please can anyone comment on the easiest way to fix this? Is a rebuild of Python necessary? Best regards David -------------- next part -------------- An HTML attachment was scrubbed... URL: From marko at pacujo.net Mon Feb 23 08:49:03 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 15:49:03 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: <87fv9wd9gw.fsf@elektro.pacujo.net> "ast" : > Is there a way to define a container object able to store some > variables so that a change of a variable make a change in this object > content ? > > I dont need this feature. It is just something I am thinking about. > > In C language, there is &A for address of A In Python, you can accomplish "&" by creating an object: >>> a = 3 >>> b = 4 >>> class AmpersandA: ... def get(self): return a ... def set(self, value): global a; a = value ... >>> class AmpersandB: ... def get(self): return b ... def set(self, value): global b; b = value ... >>> l = [ AmpersandA(), AmpersandB() ] >>> for m in l: ... m.set(7) ... >>> print(a) 7 >>> print(b) 7 >>> It's exactly like "&" in C escept it's a slight bit more verbose and the exact implementation is dependent on the name of the variable. Marko From colin.atkinson at sunderland.ac.uk Mon Feb 23 08:57:02 2015 From: colin.atkinson at sunderland.ac.uk (Colin Atkinson) Date: Mon, 23 Feb 2015 13:57:02 +0000 Subject: Python Silent Install Message-ID: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> Hi, I am deploying Python to hundreds of machines using SCCM 2012. I am using the below command to install: Msiexec /i "python.msi" TARGETDIR="C:\Program Files\Python" ALLUSERS=1 /qn Even though I am using /qn, a command prompt still appears at the end of the install for about 5-6 seconds. Is there any way to make the install totally silent? The exact version of the .msi is: python-3.4.2.amd64.msi Regards, -- Colin Atkinson IT Service Development Coordinator IT Client Services | IT Services | Business Infrastructure | Business Services University Of Sunderland Unit 1, Technology Park Chester Road Sunderland SR2 7PT T. (0191) 515 2207 E. colin.atkinson at sunderland.ac.uk W. http://itsupport.sunderland.ac.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Mon Feb 23 09:04:55 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Mon, 23 Feb 2015 14:04:55 +0000 Subject: Future of Pypy? References: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> of "Sun, 22 Feb 2015 19:20:44 +0000." <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> Message-ID: Laura Creighton wrote: >Good news -- it seems to be working fine with PyPy. >https://travis-ci.org/hugovk/Pillow/builds > >for me, not extensively tested, it just seems to be working. > >I have several pypy's floating around here, each within its own >virtualenv. If you aren't familiar with virtualenv, read all >about it here: >http://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ OK, thanks. I had to do a bit of experimenting to find something that worked. The "virtualenv" that came with Ubuntu failed in various ways if used with PyPy. But after downloading this and that via the PyPy download webpage, I found that "Squeaky's portable Linux binaries" contained 'virtualenv-pypy' which worked OK, then Pillow installed OK with "pip". It does seem that the PyPy that you get from the Ubuntu repository (and presumably as packaged in most Linux distros) is rather limited. Other than the core libraries included with the PyPy package itself, all that's in the repository for PyPy is Tk and an ImageMagick interface and that's it. CPython, on the other hand, has more repository packages than I can count -- hundreds, I think. So as a non-developer linux-user, I had just installed Python packages from the repository, rather than using pip, as would most linux users I would guess. So that's why I got the impression that PyPy had almost no library support. From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk Mon Feb 23 09:13:12 2015 From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance) Date: Mon, 23 Feb 2015 14:13:12 +0000 Subject: Future of Pypy? References: <54eb10cf$0$44508$c3e8da3$66d3cc2f@news.astraweb.com> Message-ID: Dave Cook wrote: >On 2015-02-22, Dave Farrance wrote: > >> It's still quicker to do a re-write in the more cumbersome C > >You should try Cython. I did try Cython when I was trying to figure out what to do about the slow speed. My initial attempt showed no speedup at all. The documentation told me that I needed to change the data types to special C-like types, so it seemed to me that it would become half way between Python and C and would be as cumbersome to develop as C. So at that point, I just rewrote it in C. Anyway, now that I've been told how to install libraries in PyPy, I'll probably be able to use PyPy if I need speedups in future. From rosuav at gmail.com Mon Feb 23 09:22:17 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 01:22:17 +1100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: On Tue, Feb 24, 2015 at 12:44 AM, David Aldrich wrote: > I want to use the Python 3.4 interpreter interactively, via a PuTTY ssh > session. Python is running on Centos 5. > > This stackoverflow thread: > > http://stackoverflow.com/questions/893053/python-shell-arrow-keys-do-not-work-on-remote-machine > > suggests that the problem can be fixed by installing the readline package: > > sudo apt-get install libreadline-dev apt-get is for Debian-based Linuxes, but you're running a Red Hat-based Linux. You'll be needing to use 'yum' instead, and I think it's "libreadline-devel" or "readline-devel" for the package name. (Apologies, I don't know CentOS very much. I'm more a Debian guy.) Did you compile Python from source, or did it come from the CentOS repositories? If you compiled from source, then most likely yes, you will need to add that library and rebuild; but it won't be a *full* rebuild - most of the interpreter isn't affected, so you'll be compiling just the handful of files that actually use readline. ChrisA From python.list at tim.thechases.com Mon Feb 23 09:33:47 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 23 Feb 2015 08:33:47 -0600 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: <20150223083347.7264596d@bigbox.christie.dr> On 2015-02-23 13:44, David Aldrich wrote: > I want to use the Python 3.4 interpreter interactively, via a PuTTY > ssh session. Python is running on Centos 5. > > Currently, the arrow keys do not work: [snip] > sudo apt-get install libreadline-dev > > followed by a rebuild of Python > > or > > pip install readline > > Please can anyone comment on the easiest way to fix this? Is a > rebuild of Python necessary? Depends on what you consider "easier". If you install readline, but don't want to rebuild Python, you could also try installing the rlwrap program and then invoking Python with that: bash$ rlwrap python which should intercept the inputs and handle them with some nice readline capabilities. Otherwise, just make sure that Python builds with readline support as you've read elsewhere. -tkc PS: and you mention being on CentOS but running apt-get. I believe CentOS and other Red-Hat based distros use "yum" instead of "apt-get" From jonas at wielicki.name Mon Feb 23 10:05:16 2015 From: jonas at wielicki.name (Jonas Wielicki) Date: Mon, 23 Feb 2015 16:05:16 +0100 Subject: Question on asyncio In-Reply-To: <87k2z8dagc.fsf@elektro.pacujo.net> References: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> <8761atiqvt.fsf@elektro.pacujo.net> <5c9fa6a5-f92a-42a8-a760-9477f9d8d0d2@googlegroups.com> <87k2z8dagc.fsf@elektro.pacujo.net> Message-ID: <54EB41AC.5020205@wielicki.name> On 23.02.2015 14:27, Marko Rauhamaa wrote: > pfranken85 at gmail.com: >> The corresponding call is a call to the python smbus library. It >> includes several sleeps (even though they are only about 50ms). >> Therefore I think it is worthwhile to encapsulate it into a coroutine. > > Maybe. Then you'll probably have to rewrite smbus to work asyncio style. > Those sleeps would then be done as "yield from" statements. > The manual has a word on this [1]. So I would suggest to use run_in_executor() instead of trashing and rewriting the whole smbus library. regards, jwi [1]: https://docs.python.org/3/library/asyncio-dev.html#handle-blocking-functions-correctly [2]: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.BaseEventLoop.run_in_executor -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From David.Aldrich at EMEA.NEC.COM Mon Feb 23 10:22:56 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Mon, 23 Feb 2015 15:22:56 +0000 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <20150223083347.7264596d@bigbox.christie.dr> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <20150223083347.7264596d@bigbox.christie.dr> Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B945298@EX10MBX02.EU.NEC.COM> Thanks for your replies, I will give readline a try. > PS: and you mention being on CentOS but running apt-get. I believe CentOS > and other Red-Hat based distros use "yum" instead of "apt-get" Yes, I think I need to use: yum install readline-devel Best regards David From mail at timgolden.me.uk Mon Feb 23 10:29:12 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 23 Feb 2015 15:29:12 +0000 Subject: Python Silent Install In-Reply-To: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> References: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> Message-ID: <54EB4748.6010601@timgolden.me.uk> On 23/02/2015 13:57, Colin Atkinson wrote: > I am deploying Python to hundreds of machines using SCCM 2012. I am > using the below command to install: > > Msiexec /i ?python.msi? TARGETDIR=?C:\Program Files\Python? > ALLUSERS=1 /qn > > Even though I am using /qn, a command prompt still appears at the end > of the install for about 5-6 seconds. Is there any way to make the > install totally silent? > > The exact version of the .msi is: > > python-3.4.2.amd64.msi > I haven't actually tried your command line myself, but I suspect that the final "command prompt" is actually the ensurepip step. You should be able to stop that by disabling that particular feature. Off the top of my head I don't know what the feature name is; I'll have to clone a 3.4 repo and check (the installer has changed substantially for 3.5). TJG From stefan_ml at behnel.de Mon Feb 23 10:43:41 2015 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 23 Feb 2015 16:43:41 +0100 Subject: Cython - was: Future of Pypy? In-Reply-To: References: <54eb10cf$0$44508$c3e8da3$66d3cc2f@news.astraweb.com> Message-ID: Dave Farrance schrieb am 23.02.2015 um 15:13: > Dave Cook wrote: >> On 2015-02-22, Dave Farrance wrote: >> >>> It's still quicker to do a re-write in the more cumbersome C >> >> You should try Cython. > > I did try Cython when I was trying to figure out what to do about the slow > speed. My initial attempt showed no speedup at all. The documentation > told me that I needed to change the data types to special C-like types, so > it seemed to me that it would become half way between Python and C and > would be as cumbersome to develop as C. So at that point, I just rewrote > it in C. The main selling point of Cython is that, while it gives you the speed of C if you write C-ish code (because it translates it to the obvious C code), you don't have to write that C-ish code unless you decide to do so. Right the next line, you can use a set comprehension or yield a value back from a generator. So, it's not "half way between Python and C", it actually covers both, almost entirely. (Oh, and also C++, if you feel like it.) Stefan From jldunn2000 at gmail.com Mon Feb 23 10:58:39 2015 From: jldunn2000 at gmail.com (loial) Date: Mon, 23 Feb 2015 07:58:39 -0800 (PST) Subject: Concatenate list values Message-ID: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Is there a quick way to concatenate all the values in a list into a string, except the first value? I want this to work with variable length lists. All values in list will be strings. Any help appreciated From gordon at panix.com Mon Feb 23 11:07:02 2015 From: gordon at panix.com (John Gordon) Date: Mon, 23 Feb 2015 16:07:02 +0000 (UTC) Subject: Concatenate list values References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: In <12821378-62af-4954-8b61-aa0738c5ff5c at googlegroups.com> loial writes: > Is there a quick way to concatenate all the values in a list into a string, except the first value? > I want this to work with variable length lists. > All values in list will be strings. > Any help appreciated big_string = ', '.join(my_list[1:]) -- John Gordon Imagine what it must be like for a real medical doctor to gordon at panix.com watch 'House', or a real serial killer to watch 'Dexter'. From alister.nospam.ware at ntlworld.com Mon Feb 23 11:08:31 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Mon, 23 Feb 2015 16:08:31 +0000 (UTC) Subject: Concatenate list values References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: On Mon, 23 Feb 2015 07:58:39 -0800, loial wrote: > Is there a quick way to concatenate all the values in a list into a > string, except the first value? > > I want this to work with variable length lists. > > All values in list will be strings. > > Any help appreciated ''.join(mylist[1:]) all elements must be strings (as specified) or it will cause an error -- petribar: Any sun-bleached prehistoric candy that has been sitting in the window of a vending machine too long. -- Rich Hall, "Sniglets" From __peter__ at web.de Mon Feb 23 11:09:10 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 23 Feb 2015 17:09:10 +0100 Subject: Concatenate list values References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: loial wrote: > Is there a quick way to concatenate all the values in a list into a > string, except the first value? > > I want this to work with variable length lists. > > All values in list will be strings. > > Any help appreciated >>> strings ['All', 'values', 'in', 'list', 'will', 'be', 'strings'] Build a new list without the first item: >>> strings[1:] ['values', 'in', 'list', 'will', 'be', 'strings'] Concatenate the strings: >>> "|".join(strings[1:]) 'values|in|list|will|be|strings' If you don't want a separator invoke the join method on an empty string: >>> "".join(strings[1:]) 'valuesinlistwillbestrings' From aberg010 at my.hennepintech.edu Mon Feb 23 11:14:04 2015 From: aberg010 at my.hennepintech.edu (Andrew Berg) Date: Mon, 23 Feb 2015 10:14:04 -0600 Subject: Concatenate list values In-Reply-To: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: <54EB51CC.3020206@my.hennepintech.edu> On 2015.02.23 09:58, loial wrote: > Is there a quick way to concatenate all the values in a list into a string, except the first value? > > I want this to work with variable length lists. > > All values in list will be strings. > > Any help appreciated > The tutorial covers strings and lists: https://docs.python.org/3/tutorial/introduction.html#strings And there is ample reference documentation for strings and lists: https://docs.python.org/3/library/stdtypes.html#string-methods https://docs.python.org/3/library/string.html https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range Get to know strings and lists, and this should be a trivial problem to solve. Or just copy/paste the answers others gave and be confused the next time you need to solve a simple problem. From python.list at tim.thechases.com Mon Feb 23 11:14:34 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 23 Feb 2015 10:14:34 -0600 Subject: Concatenate list values In-Reply-To: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: <20150223101434.52871fcc@bigbox.christie.dr> On 2015-02-23 07:58, loial wrote: > Is there a quick way to concatenate all the values in a list into a > string, except the first value? > > I want this to work with variable length lists. > > All values in list will be strings. Using "".join(my_list[1:]) should work. If it is an arbitrary iterable instead of a list (and thus can't be sliced), you can use "".join(itertools.islice(my_iter, 1, None)) -tkc From lac at openend.se Mon Feb 23 11:16:19 2015 From: lac at openend.se (Laura Creighton) Date: Mon, 23 Feb 2015 17:16:19 +0100 Subject: Future of Pypy? In-Reply-To: Message from Dave Farrance of "Mon, 23 Feb 2015 14:04:55 +0000." References: <54e9e236$0$13004$c3e8da3$5496439d@news.astraweb.com> <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> of "Sun, 22 Feb 2015 19:20:44 +0000." <32akealnfprmj3qha172e40c4cojk6cq56@4ax.com> Message-ID: <201502231616.t1NGGJSe031716@fido.openend.se> Arrgh! I forgot to warn you that you need a very recent version of virtualenv to work with PyPy. I am very sorry about that. Glad to see that things are working now. Laura From mail at timgolden.me.uk Mon Feb 23 11:25:06 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 23 Feb 2015 16:25:06 +0000 Subject: Python Silent Install In-Reply-To: <54EB4748.6010601@timgolden.me.uk> References: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> <54EB4748.6010601@timgolden.me.uk> Message-ID: <54EB5462.70104@timgolden.me.uk> On 23/02/2015 15:29, Tim Golden wrote: > On 23/02/2015 13:57, Colin Atkinson wrote: >> I am deploying Python to hundreds of machines using SCCM 2012. I am >> using the below command to install: >> >> Msiexec /i ?python.msi? TARGETDIR=?C:\Program Files\Python? >> ALLUSERS=1 /qn >> >> Even though I am using /qn, a command prompt still appears at the end >> of the install for about 5-6 seconds. Is there any way to make the >> install totally silent? >> >> The exact version of the .msi is: >> >> python-3.4.2.amd64.msi >> > > I haven't actually tried your command line myself, but I suspect that > the final "command prompt" is actually the ensurepip step. You should be > able to stop that by disabling that particular feature. Off the top of > my head I don't know what the feature name is; I'll have to clone a 3.4 > repo and check (the installer has changed substantially for 3.5). By quick inspection, it looks as though: msiexec ADDLOCAL="all" REMOVE="pip_feature" /i python-3.4.2.amd64.msi /qn should do the trick TJG From marko at pacujo.net Mon Feb 23 11:30:49 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 18:30:49 +0200 Subject: Question on asyncio References: <52b94569-1480-4a05-a58f-13c2cb19229a@googlegroups.com> <8761atiqvt.fsf@elektro.pacujo.net> <5c9fa6a5-f92a-42a8-a760-9477f9d8d0d2@googlegroups.com> <87k2z8dagc.fsf@elektro.pacujo.net> Message-ID: <87a904d1za.fsf@elektro.pacujo.net> Jonas Wielicki : > On 23.02.2015 14:27, Marko Rauhamaa wrote: >> pfranken85 at gmail.com: >>> The corresponding call is a call to the python smbus library. It >>> includes several sleeps (even though they are only about 50ms). >>> Therefore I think it is worthwhile to encapsulate it into a coroutine. >> >> Maybe. Then you'll probably have to rewrite smbus to work asyncio style. >> Those sleeps would then be done as "yield from" statements. >> > > The manual has a word on this [1]. So I would suggest to use > run_in_executor() instead of trashing and rewriting the whole smbus > library. Ok, that's always a fallback. I usually isolate such obnoxious function calls in a process pool. But that wouldn't be purely asyncio anymore. One of the main offenders is the database libraries. They should definitely provide nonblocking access. Marko From David.Aldrich at EMEA.NEC.COM Mon Feb 23 11:40:54 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Mon, 23 Feb 2015 16:40:54 +0000 Subject: How best to install path.py package on Centos 5? Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B945390@EX10MBX02.EU.NEC.COM> Hi I want to install path.py in my Python 3.4 environment on a Centos 5 box. My /usr/local/bin/ contains: easy_install-3.4 python3.4 etc. We are behind a proxy server. I tried this: # /usr/local/bin/easy_install-3.4 path.py Searching for path.py Reading https://pypi.python.org/simple/path.py/ Download error on https://pypi.python.org/simple/path.py/: hostname '172.29.68.1 ' doesn't match either of 'www.python.org', 'python.org', 'pypi.python.org', 'docs.python.org', 'testpypi.python.org', 'bugs.python.org', 'wiki.python.org', 'hg.python.org', 'mail.python.org', 'packaging.python.org', 'pythonhosted.org', 'www.pythonhosted.org', 'test.pythonhosted.org', 'us.pycon.org', 'id.python.org' -- Some packages may not be found! Couldn't find index page for 'path.py' (maybe misspelled?) Am I best to use pip or easy_install? If easy_install, how can I fix the above error? Thanks in advance. David -------------- next part -------------- An HTML attachment was scrubbed... URL: From colin.atkinson at sunderland.ac.uk Mon Feb 23 11:41:47 2015 From: colin.atkinson at sunderland.ac.uk (Colin Atkinson) Date: Mon, 23 Feb 2015 16:41:47 +0000 Subject: Python Silent Install In-Reply-To: <54EB5462.70104@timgolden.me.uk> References: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> <54EB4748.6010601@timgolden.me.uk> <54EB5462.70104@timgolden.me.uk> Message-ID: <91A935D75D3AF940BAD7318B212C554B019D5DDB2F@mb-01.uni.ad.sunderland.ac.uk> Fantastic! I ran the below command and it worked: Msiexec /i python.msi TARGETDIR="C:\Program Files\Python" ADDLOCAL="all" REMOVE="pip_feature" /qn Thanks for the quick response. Regards, -- Colin Atkinson IT Service Development Coordinator IT Client Services | IT Services | Business Infrastructure | Business Services University Of Sunderland Unit 1, Technology Park Chester Road Sunderland SR2 7PT T. (0191) 515 2207 E. colin.atkinson at sunderland.ac.uk W. http://itsupport.sunderland.ac.uk -----Original Message----- From: Python-list [mailto:python-list-bounces+colin.atkinson=sunderland.ac.uk at python.org] On Behalf Of Tim Golden Sent: 23 February 2015 16:25 To: python-list at python.org Subject: Re: Python Silent Install On 23/02/2015 15:29, Tim Golden wrote: > On 23/02/2015 13:57, Colin Atkinson wrote: >> I am deploying Python to hundreds of machines using SCCM 2012. I am >> using the below command to install: >> >> Msiexec /i "python.msi" TARGETDIR="C:\Program Files\Python" >> ALLUSERS=1 /qn >> >> Even though I am using /qn, a command prompt still appears at the end >> of the install for about 5-6 seconds. Is there any way to make the >> install totally silent? >> >> The exact version of the .msi is: >> >> python-3.4.2.amd64.msi >> > > I haven't actually tried your command line myself, but I suspect that > the final "command prompt" is actually the ensurepip step. You should be > able to stop that by disabling that particular feature. Off the top of > my head I don't know what the feature name is; I'll have to clone a 3.4 > repo and check (the installer has changed substantially for 3.5). By quick inspection, it looks as though: msiexec ADDLOCAL="all" REMOVE="pip_feature" /i python-3.4.2.amd64.msi /qn should do the trick TJG -- https://mail.python.org/mailman/listinfo/python-list From breamoreboy at yahoo.co.uk Mon Feb 23 12:13:54 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 23 Feb 2015 17:13:54 +0000 Subject: Concatenate list values In-Reply-To: <20150223101434.52871fcc@bigbox.christie.dr> References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> <20150223101434.52871fcc@bigbox.christie.dr> Message-ID: On 23/02/2015 16:14, Tim Chase wrote: > On 2015-02-23 07:58, loial wrote: >> Is there a quick way to concatenate all the values in a list into a >> string, except the first value? >> >> I want this to work with variable length lists. >> >> All values in list will be strings. > > Using > > "".join(my_list[1:]) > > should work. > > If it is an arbitrary iterable instead of a list (and thus can't be > sliced), you can use > > "".join(itertools.islice(my_iter, 1, None)) > > -tkc > Presumably rather more efficient for long lists as you're not taking a copy of (effectively) the entire list. Any volunteers to measure it, not mentioning any names, Mr D'Aprano? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From lac at openend.se Mon Feb 23 12:20:35 2015 From: lac at openend.se (Laura Creighton) Date: Mon, 23 Feb 2015 18:20:35 +0100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: Message from Chris Angelico of "Tue, 24 Feb 2015 01:22:17 +1100." References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: <201502231720.t1NHKZWA012537@fido.openend.se> DO NOT REBUILD PYTHON ON CENTOS! It can break the whole package management system which depends on having a particular version of python installed. If you are running Centos you need to use virtualenv to be safe. Laura From skip.montanaro at gmail.com Mon Feb 23 12:23:44 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 23 Feb 2015 11:23:44 -0600 Subject: Well, that was a head scratcher... Message-ID: I've been reworking some of the code in the platform I use at work. I'm the sole developer/maintainer/hard-core user left, so I can pretty much do what I want with it (convert modules to Cython, delete no longer used modules, etc). The platform uses PyGtk, so we use signals and other features of that system heavily. In one place, I register a generator function with gobject.idle_add: self.chunk_id = gobject.idle_add(self.read_chunk().next, priority=gobject.PRIORITY_LOW) read_chunk is (as you might expect) a generator function. Structurally, it looks like this: def read_chunk(self): count = 0 while True: try: x = other_stuff() except StopIteration: yield False else: if some other condition holds: yield False self.distribute(x) count += 1 if count % 1000 == 0: # Let other folks have the CPU... yield True I decided I needed to clean up that idle_add stuff when complete, so instead of duplicating that bit before both "yield False" statements, I wrapped the whole mess in a try/finally statement and did my cleanup in the finally clause. Thinking the "yield False" might circumvent the finally clause (silly me), I changed those statements to break statements. Everything seemed to work okay for awhile, as long as I was running in one (historical) mode, where the system only runs from historical data (read_chunk above is responsible for much of that), exiting when the historical data were exhausted. When I switched to live mode (initialize from historical data, then switch to listen for live data), the system kept crashing with a StopIteration before switching to live mode. I thought I had handled all the StopIteration exceptions, but the switch from "yield False" to "break" meant another one was raised which I wasn't catching. Switching back to the yield statement solved the problem. Took awhile to figure out what was going on. Skip From lac at openend.se Mon Feb 23 12:25:41 2015 From: lac at openend.se (Laura Creighton) Date: Mon, 23 Feb 2015 18:25:41 +0100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: Message from Chris Angelico of "Tue, 24 Feb 2015 01:22:17 +1100." References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: <201502231725.t1NHPfps013562@fido.openend.se> If you need to install a new version of python on Centos, follow these instructions. https://www.digitalocean.com/community/tutorials/how-to-set-up-python-2-7-6-and-3-3-3-on-centos-6-4 (From the embarassing things I have done as a consultant on my first day of work department ...) Laura From lac at openend.se Mon Feb 23 12:27:12 2015 From: lac at openend.se (Laura Creighton) Date: Mon, 23 Feb 2015 18:27:12 +0100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: Message from Chris Angelico of "Tue, 24 Feb 2015 01:22:17 +1100." References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: <201502231727.t1NHRCQO013874@fido.openend.se> And it is time for me to get on a train. Can somebody please, please make sure the OP doesn't shoot himself in the foot? Laura From rosuav at gmail.com Mon Feb 23 12:34:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 04:34:38 +1100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <201502231727.t1NHRCQO013874@fido.openend.se> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <201502231727.t1NHRCQO013874@fido.openend.se> Message-ID: On Tue, Feb 24, 2015 at 4:27 AM, Laura Creighton wrote: > And it is time for me to get on a train. Can somebody please, please > make sure the OP doesn't shoot himself in the foot? We don't have that capability, but we can at least make sure that there are appropriate warning signs on both the gun and the foot, saying "do not use these components together". :) Normally I'd just recommend the use of 'make altinstall' and no scariness; that's usually enough for preserving a system Python. ChrisA From pkpearson at nowhere.invalid Mon Feb 23 12:35:40 2015 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 23 Feb 2015 17:35:40 GMT Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: On Mon, 23 Feb 2015 13:55:47 +0100, ast wrote: > hi > >>>> a = 2; b = 5 >>>> Li = [a, b] >>>> >>>> Li > [2, 5] >>>> a=3 >>>> Li > [2, 5] >>>> > > Ok, a change in a or b doesn't impact Li. This works as expected > > Is there a way to define a container object able to store some variables > so that a change of a variable make a change in this object content ? > > I dont need this feature. It is just something I am thinking about. > > In C language, there is &A for address of A The word "variable" brings implications and assumptions that get people into trouble in Python. Python doesn't have variables. It has objects, and you can assign names to objects -- like sticky notes, as someone in this newsgroup helpfully pointed out long ago. This way of thinking is significantly different from the familiar "variable" mind-set of C, but looks similar enough to produce some confusion. For me, the sticky-note analogy helped a lot, and in a few days many things made more sense. -- To email me, substitute nowhere->runbox, invalid->com. From breamoreboy at yahoo.co.uk Mon Feb 23 12:37:49 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 23 Feb 2015 17:37:49 +0000 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <201502231727.t1NHRCQO013874@fido.openend.se> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <201502231727.t1NHRCQO013874@fido.openend.se> Message-ID: On 23/02/2015 17:27, Laura Creighton wrote: > And it is time for me to get on a train. Can somebody please, please > make sure the OP doesn't shoot himself in the foot? > > Laura > Nope, what with consenting adults and all that it's the best way to learn. Especially if you use a test system to start with, such as you're effectively using blanks :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Mon Feb 23 12:42:38 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 04:42:38 +1100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <201502231727.t1NHRCQO013874@fido.openend.se> Message-ID: On Tue, Feb 24, 2015 at 4:37 AM, Mark Lawrence wrote: > On 23/02/2015 17:27, Laura Creighton wrote: >> >> And it is time for me to get on a train. Can somebody please, please >> make sure the OP doesn't shoot himself in the foot? >> >> Laura >> > > Nope, what with consenting adults and all that it's the best way to learn. > Especially if you use a test system to start with, such as you're > effectively using blanks :) That's not using blanks. That's using replaceable feet. ChrisA From rgaddi at technologyhighland.invalid Mon Feb 23 13:10:20 2015 From: rgaddi at technologyhighland.invalid (Rob Gaddi) Date: Mon, 23 Feb 2015 18:10:20 +0000 (UTC) Subject: Best practice: Sharing object between different objects References: Message-ID: On Sat, 21 Feb 2015 04:15:50 -0800, pfranken85 wrote: > Hello! > > I have a best-practice question: Imagine I have several hardware devices > that I work with on the same I2C bus and I am using the python smbus > module for that purpose. The individual devices are sensors, ADC, DAC > components. As I said, I would like to derive the corresponding classes > from one common class, let's say I2CDevice, so that they can share the > same bus connection (I don't want to do a import smbus, ..., self.bus = > smbus.SMBus(1) all the time). Of course, I could just pass the the bus > reference to each instance, but I am pretty sure that there must be a > nicer way to do this. > > In particular, I imagine the following: It should be possible that I > have two classes, ADC_I2C, DAC_I2C which share the same base class. Once > I create an instance of ADC_I2C or DAC_I2C it should check whether a bus > object exists, if not, it should create one and the other class should > be able to use this bus reference as well. Do you get my point? I am > pretty sure that such a design pattern should exist, maybe also in the > reference of DB connections? Unfortunately I did not find something like > this. > > Any hints are highly appreciated! > > Thanks! So my experience doing similar work is with VMEBus, but if I were trying to do what you're doing, I'd take advantage of the fact that hardware isn't nearly as abstract as software. SMBus(1) represents three physical contiguous pieces of copper on a PCB somewhere. So I'd solve it with module level global variables. It's semi-frowned upon on software stuff because it creates an unintentional shared state between different modules, but you really HAVE a shared state, so it needs to be dealt with. So I'd put your ADC and DAC in the same package, whether all in one file or all in a package folder, and I'd give that package a: def getSMBus(busnumber): that takes care of the management of only creating each bus once. Then I'd give each class a: def __init__(busnumber): self.bus = getSMBus(busnumber) You could commonize that by deriving both classes from a common ancestor (all my VME modules derive from a common Region), but with only one piece of shared code between I don't think I'd bother. If you did, you could make the getSMBus code a classmethod just to make clear the lack of dependency on individual instances, but that starts to be code for code's sake. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From marko at pacujo.net Mon Feb 23 13:22:31 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 20:22:31 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: <8761ascwt4.fsf@elektro.pacujo.net> Peter Pearson : >>>>> a = 2; b = 5 >>>>> Li = [a, b] >>>>> Li >> [2, 5] >>>>> a=3 >>>>> Li >> [2, 5] > > [...] > > The word "variable" brings implications and assumptions that get > people into trouble in Python. Python doesn't have variables. > It has objects, and you can assign names to objects -- like sticky > notes, as someone in this newsgroup helpfully pointed out long ago. > > This way of thinking is significantly different from the familiar > "variable" mind-set of C, but looks similar enough to produce some > confusion. For me, the sticky-note analogy helped a lot, and in a > few days many things made more sense. It's no different in C: #include int main() { int a = 2, b = 5; int Li[2] = { a, b }; printf("%d %d\n", Li[0], Li[1]); a = 3; printf("%d %d\n", Li[0], Li[1]); return 0; } Outputs: 2 5 2 5 Marko From calderon.christian760 at gmail.com Mon Feb 23 13:26:06 2015 From: calderon.christian760 at gmail.com (Christian Calderon) Date: Mon, 23 Feb 2015 10:26:06 -0800 Subject: multiprocessing module best practices? Message-ID: Is there a good blog or tutorial on when it is most appropriate to use the Process vs Pool classes, and when you should subclass either of them? I have written a few programs using these tools, and each time I have to sort of make up my own design patterns as I go. I want to know what the community thinks is the most pythonic use of the multiprocessing module (I could just subclass Process every single time, why shouldn't I?) I'm also curious about when I should be importing modules in the main module to use in a Process, and when I should import the module inside the run method/target function. Certain modules just won't work unless I import it in the Process; I hate having a ton of import statements inside my functions, but I also hate having to guess-and-check just to figure out which modules I can import outside the function. And it is just kinda ugly to have import statements at multiple parts of the same piece of code, IMO. Here is an example of something I wrote that feels kinda funky (my excuse is that the code-base I was working with was incredibly funky to begin with, and I was trying to make a small corner of it nicer/actually work.): https://github.com/zack-bitcoin/augur-core/blob/master/database.py -------------- next part -------------- An HTML attachment was scrubbed... URL: From torriem at gmail.com Mon Feb 23 13:36:05 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 23 Feb 2015 11:36:05 -0700 Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: <54EB7315.5030601@gmail.com> On 02/23/2015 11:10 AM, Rob Gaddi wrote: > So I'd solve it with module level global variables. It's semi-frowned > upon on software stuff because it creates an unintentional shared state > between different modules, but you really HAVE a shared state, so it > needs to be dealt with. I would also do it with a module attribute. In my mind that's exactly the right way to go. But I disagree that it's frowned on or a bad thing. A module is a completely appropriate place to store state. In fact a module is an object, but a special one that can only be instantiated once. So as far as patterns go, a module is a singleton. Almost any time in Python you have something that you want to have exactly one instance of in your program, you don't want to define a class but rather just use a module. Any code in a module can be considered the constructor. It executes only once in your program when the module is first imported, no matter how many times its imported after that. I often use a module to store configuration that is shared across modules in my projects. From __peter__ at web.de Mon Feb 23 13:41:23 2015 From: __peter__ at web.de (Peter Otten) Date: Mon, 23 Feb 2015 19:41:23 +0100 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > Peter Pearson : > >>>>>> a = 2; b = 5 >>>>>> Li = [a, b] >>>>>> Li >>> [2, 5] >>>>>> a=3 >>>>>> Li >>> [2, 5] >> >> [...] >> >> The word "variable" brings implications and assumptions that get >> people into trouble in Python. Python doesn't have variables. >> It has objects, and you can assign names to objects -- like sticky >> notes, as someone in this newsgroup helpfully pointed out long ago. >> >> This way of thinking is significantly different from the familiar >> "variable" mind-set of C, but looks similar enough to produce some >> confusion. For me, the sticky-note analogy helped a lot, and in a >> few days many things made more sense. > > It's no different in C: > > #include > > int main() > { > int a = 2, b = 5; > int Li[2] = { a, b }; > printf("%d %d\n", Li[0], Li[1]); > a = 3; > printf("%d %d\n", Li[0], Li[1]); > return 0; > } > > Outputs: > > 2 5 > 2 5 > > > Marko The OP explicitly mentions the & operator. There's no python analog to that and the behavior shown below: $ cat pointers.c #include int main() { int a = 2, b = 5; int * Li[2] = { &a, &b }; printf("%d %d\n", *Li[0], *Li[1]); a = 3; printf("%d %d\n", *Li[0], *Li[1]); return 0; } $ gcc pointers.c $ ./a.out 2 5 3 5 From marko at pacujo.net Mon Feb 23 14:06:11 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 21:06:11 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> Message-ID: <87385wh2ho.fsf@elektro.pacujo.net> Peter Otten <__peter__ at web.de>: > The OP explicitly mentions the & operator. There's no python analog to > that and the behavior shown below: > > $ cat pointers.c > #include > > int main() > { > int a = 2, b = 5; > int * Li[2] = { &a, &b }; > printf("%d %d\n", *Li[0], *Li[1]); > a = 3; > printf("%d %d\n", *Li[0], *Li[1]); > return 0; > } > $ gcc pointers.c > $ ./a.out > 2 5 > 3 5 But there is! I just demonstrated it in a previous posting: > In Python, you can accomplish "&" by creating an object: > > >>> a = 3 > >>> b = 4 > >>> class AmpersandA: > ... def get(self): return a > ... def set(self, value): global a; a = value > ... > >>> class AmpersandB: > ... def get(self): return b > ... def set(self, value): global b; b = value > ... > >>> l = [ AmpersandA(), AmpersandB() ] > >>> for m in l: > ... m.set(7) > ... > >>> print(a) > 7 > >>> print(b) > 7 > >>> > > It's exactly like "&" in C escept it's a slight bit more verbose and the > exact implementation is dependent on the name of the variable. You can find an analogous ampersand object for any C lvalue. What I'm saying is that there's nothing special about Python's object model or variables. Guido could decide tomorrow to add a C-esque "&" operator to Python without breaking a single existing Python program. The Python compiler would simply generate code like above. Marko From rosuav at gmail.com Mon Feb 23 14:17:48 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 06:17:48 +1100 Subject: list storing variables In-Reply-To: <87385wh2ho.fsf@elektro.pacujo.net> References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> <87385wh2ho.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 24, 2015 at 6:06 AM, Marko Rauhamaa wrote: > What I'm saying is that there's nothing special about Python's object > model or variables. Guido could decide tomorrow to add a C-esque "&" > operator to Python without breaking a single existing Python program. > The Python compiler would simply generate code like above. Actually, there's a fairly huge difference. All you've done is create a thing which can set a global... you don't have any way to pass pointers around, like you can in C. (And that's even without getting into pointer *arithmetic*.) You have to construct a dedicated "thing" in your source code for every pointer you want to be able to work with. And if you try to make a single generic "Ampersand" class, what you'll find is that you're working with namespaces, not addresses. So Python's name-binding model really is fundamentally different from C's stuff-in-memory model. ChrisA From ian.g.kelly at gmail.com Mon Feb 23 14:29:33 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 23 Feb 2015 12:29:33 -0700 Subject: list storing variables In-Reply-To: <87385wh2ho.fsf@elektro.pacujo.net> References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> <87385wh2ho.fsf@elektro.pacujo.net> Message-ID: On Mon, Feb 23, 2015 at 12:06 PM, Marko Rauhamaa wrote: > But there is! I just demonstrated it in a previous posting: > >> In Python, you can accomplish "&" by creating an object: >> >> >>> a = 3 >> >>> b = 4 >> >>> class AmpersandA: >> ... def get(self): return a >> ... def set(self, value): global a; a = value >> ... >> >>> class AmpersandB: >> ... def get(self): return b >> ... def set(self, value): global b; b = value >> ... >> >>> l = [ AmpersandA(), AmpersandB() ] >> >>> for m in l: >> ... m.set(7) >> ... >> >>> print(a) >> 7 >> >>> print(b) >> 7 >> >>> >> >> It's exactly like "&" in C escept it's a slight bit more verbose and the >> exact implementation is dependent on the name of the variable. "Sure, you can have list comprehensions in C! Just write a function whose arguments are two other functions and an input array." Obviously one can use any Turing-complete language to emulate features of any other Turing-complete language, but I think the point is that there is no syntactic support for it. From ethan at stoneleaf.us Mon Feb 23 14:39:05 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 23 Feb 2015 11:39:05 -0800 Subject: Future of Pypy? In-Reply-To: <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54EB81D9.8070309@stoneleaf.us> On 02/22/2015 11:41 PM, Steven D'Aprano wrote: > If you want *CPython* to work without a GIL, well, are you volunteering to > do the work? It is a massive job, and the core devs aren't terribly > interested. Probably because they understand that the GIL is not often an > unsurmountable problem in real life. That brings us to the point I was > making: I believe, and the core devs appear to think the same, that the > *actual* number of people who would benefit from CPython losing the GIL is > not large enough to justify the work it would require. If memory serves, the first and primary point to losing the GIL is that single-threaded code must run just as fast (or at least close to just as fast) as the GIL version, and previous attempts have all badly failed that requisite. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From sohcahtoa82 at gmail.com Mon Feb 23 15:02:00 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Mon, 23 Feb 2015 12:02:00 -0800 (PST) Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: On Monday, February 23, 2015 at 10:45:59 AM UTC-8, Michael Torrie wrote: > On 02/23/2015 11:10 AM, Rob Gaddi wrote: > > So I'd solve it with module level global variables. It's semi-frowned > > upon on software stuff because it creates an unintentional shared state > > between different modules, but you really HAVE a shared state, so it > > needs to be dealt with. > > I would also do it with a module attribute. In my mind that's exactly > the right way to go. > > But I disagree that it's frowned on or a bad thing. A module is a > completely appropriate place to store state. In fact a module is an > object, but a special one that can only be instantiated once. So as far > as patterns go, a module is a singleton. Almost any time in Python you > have something that you want to have exactly one instance of in your > program, you don't want to define a class but rather just use a module. > > Any code in a module can be considered the constructor. It executes only > once in your program when the module is first imported, no matter how > many times its imported after that. > > I often use a module to store configuration that is shared across > modules in my projects. That behavior always trips me up. My intuition tells me that every time you import a module, it re-runs the code in the module. So if I had a simple module named myModule.py that had a single line with `myInt = 1`, then I would *EXPECT* this behavior: >>> import myModule >>> myModule.myInt 1 >>> myModule.myInt = 2 >>> myModule.myInt 2 >>> import myModule >>> myModule.myInt 1 It seems slightly counter-intuitive, but the *ACTUAL* behavior (where the final line prints 2) does allow for something nice: Easy use of global variables. What's REALLY interesting is that this happens: >>> import myModule >>> myModule.myInt 1 >>> myModule.myInt = 2 >>> myModule.myInt 2 >>> del myModule >>> import myModule >>> myModule.myInt 2 I would REALLY expect that deleting the module object and then re-importing would reset that variable. And then... >>> del myModule.myInt >>> del myModule >>> import myModule >>> myModule.myInt Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'myInt' So yeah...modules are really only loaded once unless you use reload/imp.reload/importlib.reload. But yeah, it seems slightly counter-intuitive to me at first, but certainly has its uses. From breamoreboy at yahoo.co.uk Mon Feb 23 15:13:31 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 23 Feb 2015 20:13:31 +0000 Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: On 23/02/2015 20:02, sohcahtoa82 at gmail.com wrote: > On Monday, February 23, 2015 at 10:45:59 AM UTC-8, Michael Torrie wrote: >> On 02/23/2015 11:10 AM, Rob Gaddi wrote: >>> So I'd solve it with module level global variables. It's semi-frowned >>> upon on software stuff because it creates an unintentional shared state >>> between different modules, but you really HAVE a shared state, so it >>> needs to be dealt with. >> >> I would also do it with a module attribute. In my mind that's exactly >> the right way to go. >> >> But I disagree that it's frowned on or a bad thing. A module is a >> completely appropriate place to store state. In fact a module is an >> object, but a special one that can only be instantiated once. So as far >> as patterns go, a module is a singleton. Almost any time in Python you >> have something that you want to have exactly one instance of in your >> program, you don't want to define a class but rather just use a module. >> >> Any code in a module can be considered the constructor. It executes only >> once in your program when the module is first imported, no matter how >> many times its imported after that. >> >> I often use a module to store configuration that is shared across >> modules in my projects. > > That behavior always trips me up. My intuition tells me that every time you import a module, it re-runs the code in the module. So if I had a simple module named myModule.py that had a single line with `myInt = 1`, then I would *EXPECT* this behavior: > >>>> import myModule >>>> myModule.myInt > 1 >>>> myModule.myInt = 2 >>>> myModule.myInt > 2 >>>> import myModule >>>> myModule.myInt > 1 > > It seems slightly counter-intuitive, but the *ACTUAL* behavior (where the final line prints 2) does allow for something nice: Easy use of global variables. > > What's REALLY interesting is that this happens: > >>>> import myModule >>>> myModule.myInt > 1 >>>> myModule.myInt = 2 >>>> myModule.myInt > 2 >>>> del myModule >>>> import myModule >>>> myModule.myInt > 2 > > I would REALLY expect that deleting the module object and then re-importing would reset that variable. And then... > >>>> del myModule.myInt >>>> del myModule >>>> import myModule >>>> myModule.myInt > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'module' object has no attribute 'myInt' > > So yeah...modules are really only loaded once unless you use reload/imp.reload/importlib.reload. > > But yeah, it seems slightly counter-intuitive to me at first, but certainly has its uses. > Rule 1) Don't expect anything of Python, always read the docs. Rule 2) If in doubt always refer to rule 1) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marko at pacujo.net Mon Feb 23 15:25:06 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 22:25:06 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> <87385wh2ho.fsf@elektro.pacujo.net> Message-ID: <87oaokfk9p.fsf@elektro.pacujo.net> Chris Angelico : > On Tue, Feb 24, 2015 at 6:06 AM, Marko Rauhamaa wrote: >> What I'm saying is that there's nothing special about Python's object >> model or variables. Guido could decide tomorrow to add a C-esque "&" >> operator to Python without breaking a single existing Python program. >> The Python compiler would simply generate code like above. > > Actually, there's a fairly huge difference. All you've done is create > a thing which can set a global... you don't have any way to pass > pointers around, like you can in C. That AmpersandA() object *is* a first-class "pointer" object. It can be passed around freely just like &A in C. > You have to construct a dedicated "thing" in your source code for > every pointer you want to be able to work with. Yes. Python doesn't have language-level support for the construct, but it could be added without a change in the data model. > And if you try to make a single generic "Ampersand" class, what > you'll find is that you're working with namespaces, not addresses. I wasn't trying to make a single generic Ampersand class. > So Python's name-binding model really is fundamentally different from > C's stuff-in-memory model. I still fail to see the fundamental difference. The main difference between the "old" languages and the "new" languages is the lack of "boxing" in the "new" languages. In C terms, Python's "." is C's "->". Marko From marko at pacujo.net Mon Feb 23 15:38:40 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Feb 2015 22:38:40 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> <8761ascwt4.fsf@elektro.pacujo.net> <87385wh2ho.fsf@elektro.pacujo.net> Message-ID: <87k2z8fjn3.fsf@elektro.pacujo.net> Ian Kelly : > Obviously one can use any Turing-complete language to emulate features > of any other Turing-complete language, but I think the point is that > there is no syntactic support for it. While my "contribution" was made tongue-in-cheek, there's a grain of truth in every joke. First of all, this is not emulating a Turing-complete language. It is demonstrating differences in the data model. The differences aren't in the supposed absence of variables or memory slots. Python has variables that are memory slots, just like C or Java. You are right that Python doesn't have syntactic support for "&". However, that has little bearing to the underlying data model. Python doesn't suffer from the absence of "&" as much as Java does, for example, because its main use case is returning multiple values. Tuples handle that just fine -- although, look at this snippet: def put_left(self, key, value): if self.left is None: self.left = Entry(self, key, value) self.balance -= 1 return self, self.right is None, self.left, True self.left, grown, entry, fresh = self.left.put(key, value) if not grown: return self, False, entry, fresh self.balance -= 1 if self.balance != -2: return self, self.balance == -1, entry, fresh return self.rotate_right(), False, entry, fresh There are other, rarer use cases where idioms such as my AmpersandA class need to be employed. But, thanks to the existence of closures, no tears are shed. Marko From ian.g.kelly at gmail.com Mon Feb 23 15:39:52 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 23 Feb 2015 13:39:52 -0700 Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: On Mon, Feb 23, 2015 at 1:02 PM, wrote: > What's REALLY interesting is that this happens: > >>>> import myModule >>>> myModule.myInt > 1 >>>> myModule.myInt = 2 >>>> myModule.myInt > 2 >>>> del myModule >>>> import myModule >>>> myModule.myInt > 2 > > I would REALLY expect that deleting the module object and then re-importing would reset that variable. Even though you deleted the module locally, it's still referenced in the sys.modules cache (as well as in any other place where it might have been imported). That's the place you need to delete it from if you really want to re-execute it. >>> import myModule >>> myModule.myInt 1 >>> myModule.myInt = 2 >>> myModule.myInt 2 >>> import sys >>> del sys.modules['myModule'] >>> import myModule >>> myModule.myInt 1 From ginolampascione at gmx.com Mon Feb 23 15:47:39 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Mon, 23 Feb 2015 12:47:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: -- redacted -- From ginolampascione at gmx.com Mon Feb 23 15:48:02 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Mon, 23 Feb 2015 12:48:02 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <192cf819-ba4d-44f0-991b-7390662057e7@googlegroups.com> -- redacted -- From ginolampascione at gmx.com Mon Feb 23 15:51:16 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Mon, 23 Feb 2015 12:51:16 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <27aa1def-11ea-4ab9-b02b-7012bfa7d41a@googlegroups.com> -- redacted -- From ginolampascione at gmx.com Mon Feb 23 15:52:22 2015 From: ginolampascione at gmx.com (CARMELO PIGNATRACOZZA DE ROSARIO ARGENTINA) Date: Mon, 23 Feb 2015 12:52:22 -0800 (PST) Subject: -- redacted -- In-Reply-To: <83af10d1-4ece-4480-b42d-270d6263bf7d@googlegroups.com> References: <83af10d1-4ece-4480-b42d-270d6263bf7d@googlegroups.com> Message-ID: -- redacted -- From toby at tobiah.org Mon Feb 23 16:00:51 2015 From: toby at tobiah.org (Tobiah) Date: Mon, 23 Feb 2015 13:00:51 -0800 Subject: new.py and having '.' in PYTHONPATH Message-ID: Was working along, and wasted some time on this. Wanted to try something on my script, so I copied it to 'new.py'. It wouldn't run, and neither would the one I copied from! Traceback (most recent call last): File "a2z.py", line 6, in from suds.client import Client File "build/bdist.linux-i686/egg/suds/__init__.py", line 154, in File "build/bdist.linux-i686/egg/suds/client.py", line 30, in File "build/bdist.linux-i686/egg/suds/sudsobject.py", line 25, in ImportError: cannot import name classobj I finally figured out that the presence of new.py in my directory was breaking things. '.' is in my PYTHONPATH, so maybe suds was looking for some other new.py. Anyway, it raises the question as to whether having '.' in the PYTHONPATH is at all a sane thing to do. Thanks, Tobiah From get2sada at gmail.com Mon Feb 23 16:09:52 2015 From: get2sada at gmail.com (Sada Shirol) Date: Mon, 23 Feb 2015 16:09:52 -0500 Subject: How to apply python patch for issue9729 Message-ID: ?Hi everyone: Below is the error while I try to connect to Cassandra database with SSL-enable Python driver: raise NoHostAvailable("Unable to connect to any servers", errors) cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'IP Address': TypeError("'member_descriptor' object is not callable",)}) Upon some research found that I need to apply below patch: http://bugs.python.org/issue9729 There are three version for patch, I assume I should apply the latest version of the three files - issue9729-3.patch: issue9729.patch Dated: 2010-09-14 06:28 issue9729-2.patch Dated:2010-09-14 13:06 issue9729-3.patch Dated:2010-09-14 13:36 How do I apply a patch to python 2.6.6 on Linux? I tried to use the below patch tool patch -p1 --dry-run < name.patch what should be replace for --dry-run in case of python? Below is my OS: OS: Linux ------.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Any input is much appreciated. Thank You, -------------- next part -------------- An HTML attachment was scrubbed... URL: From torriem at gmail.com Mon Feb 23 16:10:07 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 23 Feb 2015 14:10:07 -0700 Subject: Best practice: Sharing object between different objects In-Reply-To: References: Message-ID: <54EB972F.1040102@gmail.com> On 02/23/2015 01:02 PM, sohcahtoa82 at gmail.com wrote: > That behavior always trips me up. My intuition tells me that every > time you import a module, it re-runs the code in the module. So if I > had a simple module named myModule.py that had a single line with > `myInt = 1`, then I would *EXPECT* this behavior: Basically your intuition is wrong, but I'm not really sure why you think this in the first place, especially if you're familiar with design patterns. A module is a singleton. Plain and simple. If you expect a module to reinstantiate something every time you import it, then what you want is a class, not a module. Even if you think of modules like C header files, as far as use goes, you wouldn't be too far wrong. Generally speaking header files are processed only once no matter how many times they are included (at least the common ones with #ifdef fences in them). I can see how coming from PHP would trip you up, though. Of course I never got the hang of include vs include_once vs require vs require_once. From ben+python at benfinney.id.au Mon Feb 23 17:03:18 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 24 Feb 2015 09:03:18 +1100 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: <85ioes70bd.fsf@benfinney.id.au> "ast" writes: > Ok, a change in a or b doesn't impact Li. This works as expected Because a container stores references to objects. The other references (such as names) that object might have are not stored. > Is there a way to define a container object able to store some > variables so that a change of a variable make a change in this object > content ? As has been pointed out, Python does not have ?variables? in the sense of a named box containing a value. What Python does have is a mapping type. You can store a key ? value mapping for each value you want to reference later. >>> a = 2; b = 5 >>> foo = {'a': a, 'b': b} >>> foo {'b': 5, 'a': 2} >>> foo['a'] = 3 >>> foo {'b': 5, 'a': 3} You appear, though, to want this to somehow automatically track the changes in name bindings. No, there's no way to do that except explicitly coding it yourself. > In C language, there is &A for address of A There is no ?address of a value? concept in Python. You access a value by some reference, either a name or an item in a collection. When a reference changes to reference some different value, other references are not affected. -- \ ?All opinions are not equal. Some are a very great deal more | `\ robust, sophisticated, and well supported in logic and argument | _o__) than others.? ?Douglas Adams | Ben Finney From sohcahtoa82 at gmail.com Mon Feb 23 17:07:29 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Mon, 23 Feb 2015 14:07:29 -0800 (PST) Subject: new.py and having '.' in PYTHONPATH In-Reply-To: References: Message-ID: <72425a30-d3bb-4e7a-86ea-e49bf7efbac3@googlegroups.com> On Monday, February 23, 2015 at 1:01:07 PM UTC-8, Tobiah wrote: > Was working along, and wasted some time on this. > > Wanted to try something on my script, so I copied > it to 'new.py'. It wouldn't run, and neither would > the one I copied from! > > Traceback (most recent call last): > File "a2z.py", line 6, in > from suds.client import Client > File "build/bdist.linux-i686/egg/suds/__init__.py", line 154, in > > File "build/bdist.linux-i686/egg/suds/client.py", line 30, in > File "build/bdist.linux-i686/egg/suds/sudsobject.py", line 25, in > ImportError: cannot import name classobj > > > I finally figured out that the presence of new.py in > my directory was breaking things. '.' is in my > PYTHONPATH, so maybe suds was looking for some > other new.py. > > Anyway, it raises the question as to whether having '.' in the > PYTHONPATH is at all a sane thing to do. > > Thanks, > > Tobiah There shouldn't be a need to have '.' in your PYTHONPATH. If your current working directory is /home/me, and you have a module in /home/me/myModule.py, then opening up the interpreter and typing `import myModule` should work fine without '.' in your PYTHONPATH. Are you running Python 2.x? There was a module named "new" in 2.x that was removed in 3. You might be dealing with a conflict when you call your own module 'new.py'. From ethan at stoneleaf.us Mon Feb 23 17:14:33 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 23 Feb 2015 14:14:33 -0800 Subject: new.py and having '.' in PYTHONPATH In-Reply-To: References: Message-ID: <54EBA649.8030401@stoneleaf.us> On 02/23/2015 01:00 PM, Tobiah wrote: > Anyway, it raises the question as to whether having '.' in the > PYTHONPATH is at all a sane thing to do. The current directory is added to sys.path /only/ for the interactive interpreter. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ian.g.kelly at gmail.com Mon Feb 23 17:29:05 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 23 Feb 2015 15:29:05 -0700 Subject: new.py and having '.' in PYTHONPATH In-Reply-To: <54EBA649.8030401@stoneleaf.us> References: <54EBA649.8030401@stoneleaf.us> Message-ID: On Mon, Feb 23, 2015 at 3:14 PM, Ethan Furman wrote: > On 02/23/2015 01:00 PM, Tobiah wrote: > >> Anyway, it raises the question as to whether having '.' in the >> PYTHONPATH is at all a sane thing to do. > > The current directory is added to sys.path /only/ for the interactive interpreter. When executing a file, the directory containing the file is added to sys.path instead, which amounts to the same thing if the file that you're executing is in your current working directory. From davecook at nowhere.net Mon Feb 23 18:23:00 2015 From: davecook at nowhere.net (Dave Cook) Date: 23 Feb 2015 23:23:00 GMT Subject: Future of Pypy? References: <54eb10cf$0$44508$c3e8da3$66d3cc2f@news.astraweb.com> Message-ID: <54ebb654$0$14741$b1db1813$46decd27@news.astraweb.com> On 2015-02-23, Dave Farrance wrote: > I did try Cython when I was trying to figure out what to do about the slow > speed. My initial attempt showed no speedup at all. The documentation > told me that I needed to change the data types to special C-like types, so > it seemed to me that it would become half way between Python and C and > would be as cumbersome to develop as C. So at that point, I just rewrote > it in C. Right, you have add type info to get the speedup. This can be done as needed. What Cython provides is a transparent interface between the C compiler and Python, without the need for any wrapper code. I'm curious about the process you use to interface to C that is less cumbersome. Dave Cook From no.email at nospam.invalid Mon Feb 23 19:11:37 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 16:11:37 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> Message-ID: <87bnkkb22u.fsf@jester.gateway.pace.com> Ryan Stuart writes: > Threads can also share read-only data and you can pass arbitrary > objects (such as code callables that you want the other thread to > execute--this is quite useful) through Queue.Queue. I don't think > you can do that with the multiprocessing module. > > These things might be convenient but they are error prone for the > reasons pointed out. I don't see the error-proneness since nothing there seems to set off mutation of shared data. > Also, the majority can be achieved via the process approach. For > example, using fork to take a copy of the current process (including > the heap) you want to use will give you access to any callables on the > heap. What if you want to dynamically construct a callable and send it to another process? > Even if you are extra careful to not touch any shared state in your > code, you can almost be guaranteed that code higher up the stack, like > malloc for example, *will* be using shared state. This isn't the 1980's any more--any serious malloc implementation these days is thread safe. People write multi-threaded C programs all the time and those programs use malloc in more than one thread. > Even if you aren't sharing state in your code directly, code higher up > the stack will be sharing state. That is the whole point of a thread, > that's what they were invented for. Using threads safely might well > be impossible much less verifiable. You're basically saying it's impossible to write a reliable operating system, since OS's by nature have to do that stuff. Of course there are verified OS's, and some of the early pioneers in concurrency were the same guys who worked in program verification, e.g. Dijkstra's semaphores. Even Erlang uses data sharing under the hood (ETS tables and large binaries) though their API makes it look like the data is copied between processes. What I'd say is that multi-threaded programs tend to have miniature OS's inside them, so it helps to have had some exposure to OS implementation techniques if you're going to write this kind of code. But if you've had that exposure then it all becomes less scary. > So when there are other options that are just as viable/functional, > result in far less risk and are often much quicker to implement > correctly, why wouldn't you use them? I should give the multiprocessing module a try sometime (haven't used it so far because it's relatively new and I'm comfortable with threads). It has the disadvantages that I noted, though. > If it were easy to use threads in a verifiably safe manner, then there > probably wouldn't be a GIL. Nah, the GIL is just a CPython artifact. As Steven says, IronPython and Jython don't have GIL's. Java has no GIL, OCaml has no GIL, GHC has no GIL, etc. Someone made a CPython version with no GIL some years ago and it worked fine and it got a speedup on multiple cores. The only problem was that on a single core, it was significantly slower than regular CPython, specifically because of the overhead of having to lock all the refcount updates, so it was considered a failure. Laura Creighton may have more to say about this, but I've been under the impression that the main obstacle to getting rid of the CPython GIL is the refcount system (which is also easy to make mistakes with, by the way). That's why I was surprised to hear that PyPy has a GIL. From rosuav at gmail.com Mon Feb 23 19:12:02 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 11:12:02 +1100 Subject: How to apply python patch for issue9729 In-Reply-To: References: Message-ID: On Tue, Feb 24, 2015 at 8:09 AM, Sada Shirol wrote: > Upon some research found that I need to apply below patch: > > http://bugs.python.org/issue9729 > > How do I apply a patch to python 2.6.6 on Linux? Hmm. You have a bit of a problem there: The patch wasn't designed to apply to 2.6. So you may find that it doesn't apply at all, or needs some tweaking. But the basic plan would be: Grab the source code for either 2.6.9 or the exact 2.6.6 that was built for your system, build it and run the test suite (just to make sure), then apply the patch, compile again, and rerun the test suite. However, that's going to be a lot of hassle, and may well break stuff on your system. What I'd recommend is this: Don't try to patch your existing 2.6.6, but instead, grab yourself a Python 2.7 as a secondary. See if you can simply install a package named python2.7 (apt-get install python2.7, or yum install python2.7), or if that fails, grab the source code (either the 2.7.8 release or the latest from the 2.7 branch in Mercurial) and compile it. Either way, it'd be an *alternate* Python; when you simply run "python", you'll continue to run the unpatched 2.6.6, but if you explicitly run "python2.7", you'll get the newer version. That way, you can't break anything that, wittingly or unwittingly, depends on the behaviour of 2.6; but if you want the patch, you can easily call it up. And then, of course, test. Test test test. But you knew that part already. :) ChrisA From rosuav at gmail.com Mon Feb 23 19:14:57 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 11:14:57 +1100 Subject: Best practice: Sharing object between different objects In-Reply-To: <54EB972F.1040102@gmail.com> References: <54EB972F.1040102@gmail.com> Message-ID: On Tue, Feb 24, 2015 at 8:10 AM, Michael Torrie wrote: > On 02/23/2015 01:02 PM, sohcahtoa82 at gmail.com wrote: >> That behavior always trips me up. My intuition tells me that every >> time you import a module, it re-runs the code in the module. So if I >> had a simple module named myModule.py that had a single line with >> `myInt = 1`, then I would *EXPECT* this behavior: > > I can see how coming from PHP would trip you up, though. Of course I > never got the hang of include vs include_once vs require vs require_once. If you think of "import" as "go and grab this file of code and run it" (like PHP's include or a preprocessor #include directive), then yes, you would expect that. But think of it, instead, as "give me access to this feature". If that feature doesn't yet exist, Python will go and fetch it up; but if it does, yay! You now have access, really quickly and easily. ChrisA From torriem at gmail.com Mon Feb 23 19:24:55 2015 From: torriem at gmail.com (Michael Torrie) Date: Mon, 23 Feb 2015 17:24:55 -0700 Subject: [OT] - BASIC is still not a bad choice, was Re: Future of Pypy? In-Reply-To: <5qpjea5125jcq3d18nvn91hb9mjun95dif@4ax.com> References: <5qpjea5125jcq3d18nvn91hb9mjun95dif@4ax.com> Message-ID: <54EBC4D7.2020907@gmail.com> On 02/22/2015 08:30 AM, Dave Farrance wrote: > I started work 1980ish, had an early 6502-based home computer, and my then > place of work had some 6502-based Pet computers, so I gained the ability > to quickly write BASIC programs as an engineering aid. Later, when BASIC > dropped into obscurity, I switched to C and C++, although I always found > that cumbersome compared to the old BASIC. Later still, when I found that > my Google queries for code examples started returning more Python than C, > I tried that -- and discovered that Python was like BASIC, only better. I know BASIC is a hiss and a byword for many folks, and certainly it's off topic here. But modern BASIC, particarly in the form of the FreeBASIC[1] compiler, is very much alive and well on all modern platforms (including Windows, Mac, Linux, and Arm devices). BASIC today appears to be every bit as structured as any modern language. It's relatively easy to translate C header files into FreeBASIC include files, but it is tedious. I would love to have a FreeBASIC translation of the libpython header files and then I could write Python extensions in FreeBASIC. Back on topic, have you tried using Cython to compile python modules that need a speedup into a binary module? [1] http://www.freebasic.net/ From rosuav at gmail.com Mon Feb 23 19:31:42 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 11:31:42 +1100 Subject: Future of Pypy? In-Reply-To: <87bnkkb22u.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> Message-ID: On Tue, Feb 24, 2015 at 11:11 AM, Paul Rubin wrote: > What if you want to dynamically construct a callable and send it to > another process? I'm not sure what that would actually mean. Do you try to construct it out of code that already exists in the other process? Are you passing actual code to the other process? Does the callable, when called, actually execute in the calling process? And what about its context - its globals, and possibly nonlocals (if it's a closure)? ChrisA From ryan.stuart.85 at gmail.com Mon Feb 23 19:35:31 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Tue, 24 Feb 2015 00:35:31 +0000 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> Message-ID: On Tue Feb 24 2015 at 10:15:40 AM Paul Rubin wrote: > > I don't see the error-proneness since nothing there seems to set off > mutation of shared data. > I'm not sure what else to say really. It's just a fact of life that Threads by definition run in the same memory space and hence always have the possibility of nasty unforeseen problems. They are unforeseen because it is extremely difficult (maybe impossible?) to try and map out and understand all the different possible mutations to state. Sure, your code might not be making any mutations (that you know of), but malloc definitely is [1], and that's just the tip of the iceberg. Other things like buffers for stdin and stdout, DNS resolution etc. all have the same issue. I have no doubt someone can come up with a scenario where they need to use threads. I can't come up with one myself, but maybe someone else can. But in the work I have done, processes have sufficed - even for the example of dynamic callables you gave. To borrow from the original article I linked - "Nevertheless I still think it?s a bad idea to make things harder for ourselves if we can avoid it." Cheers [1] Line 70 of glibc malloc - https://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/arena.c;h=8af51f05eb376ae2ba07e99c8c766a8ae8af425b;hb=bdf1ff052a8e23d637f2c838fa5642d78fcedc33#l70 > > > Also, the majority can be achieved via the process approach. For > > example, using fork to take a copy of the current process (including > > the heap) you want to use will give you access to any callables on the > > heap. > > What if you want to dynamically construct a callable and send it to > another process? > > > Even if you are extra careful to not touch any shared state in your > > code, you can almost be guaranteed that code higher up the stack, like > > malloc for example, *will* be using shared state. > > This isn't the 1980's any more--any serious malloc implementation these > days is thread safe. People write multi-threaded C programs all the > time and those programs use malloc in more than one thread. > > > Even if you aren't sharing state in your code directly, code higher up > > the stack will be sharing state. That is the whole point of a thread, > > that's what they were invented for. Using threads safely might well > > be impossible much less verifiable. > > You're basically saying it's impossible to write a reliable operating > system, since OS's by nature have to do that stuff. Of course there are > verified OS's, and some of the early pioneers in concurrency were the > same guys who worked in program verification, e.g. Dijkstra's > semaphores. Even Erlang uses data sharing under the hood (ETS tables > and large binaries) though their API makes it look like the data is > copied between processes. > > What I'd say is that multi-threaded programs tend to have miniature OS's > inside them, so it helps to have had some exposure to OS implementation > techniques if you're going to write this kind of code. But if you've > had that exposure then it all becomes less scary. > > > So when there are other options that are just as viable/functional, > > result in far less risk and are often much quicker to implement > > correctly, why wouldn't you use them? > > I should give the multiprocessing module a try sometime (haven't used it > so far because it's relatively new and I'm comfortable with threads). > It has the disadvantages that I noted, though. > > > If it were easy to use threads in a verifiably safe manner, then there > > probably wouldn't be a GIL. > > Nah, the GIL is just a CPython artifact. As Steven says, IronPython and > Jython don't have GIL's. Java has no GIL, OCaml has no GIL, GHC has no > GIL, etc. Someone made a CPython version with no GIL some years ago and > it worked fine and it got a speedup on multiple cores. The only problem > was that on a single core, it was significantly slower than regular > CPython, specifically because of the overhead of having to lock all the > refcount updates, so it was considered a failure. Laura Creighton may > have more to say about this, but I've been under the impression that the > main obstacle to getting rid of the CPython GIL is the refcount system > (which is also easy to make mistakes with, by the way). That's why I > was surprised to hear that PyPy has a GIL. > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Feb 23 20:23:24 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Feb 2015 20:23:24 -0500 Subject: How to apply python patch for issue9729 In-Reply-To: References: Message-ID: On 2/23/2015 4:09 PM, Sada Shirol wrote: > ?Hi everyone: Below is the error while I try to connect to Cassandra > database with SSL-enable Python driver: > raise NoHostAvailable("Unable to connect to any servers", errors) > cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', > {'IP Address': TypeError("'member_descriptor' object is not callable",)}) > Upon some research found that I need to apply below patch: > http://bugs.python.org/issue9729 > There are three version for patch, I assume I should apply the latest > version of the three files - issue9729-3.patch: > issue9729.patch Dated: 2010-09-14 06:28 > issue9729-2.patch Dated:2010-09-14 13:06 > issue9729-3.patch Dated:2010-09-14 13:36 You should apply the version actually committed (or something based on it). https://hg.python.org/cpython/rev/7f99ac53014a/ This may be the same as the -3 patch but might have been tweaked. > How do I apply a patch to python 2.6.6 on Linux? > I tried to use the below patch tool > patch -p1 --dry-run < name.patch > what should be replace for --dry-run in case of python? > Below is my OS: > OS: Linux ------.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 > x86_64 x86_64 GNU/Linux You should remove the diff for Misc/NEWS, as it is not in your installation. You can also do without the addition to Lib/test/test_ssl.py, or add it by hand. If you can bet a generic patch program to just patch Lib/ssl.py, great. But you could do this by hand. However, I second the suggestion to install 2.7, especially if you can 2.7.9, which has this patch, even if you have to download the source and compile. 2.7 is more compatible with 3.x than 2.6, and 2.7.9 has 100s of little bug fixes compared to 2.6.6. -- Terry Jan Reedy From no.email at nospam.invalid Mon Feb 23 20:47:32 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 17:47:32 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: <8761asaxmz.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > Deadlocks, livelocks, and the simple fact that debugging threaded code is > much, much, much harder than debugging single-thread code. Deadlocks and livelocks can happen in multi-process code just like with threads. Debugging threaded code hasn't been too bad a problem for me so far. The basic tactic is use the logging module a lot (it is thread safe), log the i/o events coming into the system if you observe misbehaviour, and play the log back through your test harness to reproduce the problem and debug it by normal means. You have to do something like that anyway, if the indeterminacy in the system comes from the unpredictable ordering of external i/o events. IMHO you'd get basicaly similar bugs with other concurrency mechanisms. Other programs like Postgres are written in dangerous languages like C and use millions of LOC and vast numbers of fine-grained locks and are still considered very reliable. Python threads communicating through queues has to be a breeze by comparison. Compared to what Linux kernel hackers have to deal with, it's not even on the same planet. This book is amazing: https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html > No. You [generic you] sux, for not investigating whether multiprocessing > will do the job. It even has the same public API as threads. I will give multiprocessing a try next time the matter comes up (it hasn't always been available) but it doesn't seem as flexible. E.g. say I have an employee handling thread (untested pseudocode, forgive errors): def employee_loop(queue): while True: func, args = queue.get() func(*args) def adjust_salary(person, update_func): salaries[person] = update_func(salaries[person]) threading.Thread(target=employee_loop, args=[employee_request_queue]).start() Now in another thread, I want to give you a 50% raise: employee_request_queue.put( (adjust_salary, ('Steven', lambda old_salary: old_salary * 1.5))) Can I do that with multiprocessing without a bunch more boilerplate? > Even in GIL-less Python, two threads aren't twice as fast as one > thread. So there comes a point of diminishing returns Well if two threads are 1.5x as fast as one thread, that's a win. > Python does have the same capabilities. Jython and IronPython aren't > different languages, they are Python. Oh ok. I think of Jython as a Java thing and IronPython as a Windows thing and I don't want to deal with the monstrous JVM or CLR systems. So I unconsciously didn't think of them as Python. I do think of PyPy as Python. > If you want *CPython* to work without a GIL, well, are you > volunteering to do the work? It is a massive job, and the core devs > aren't terribly interested. It's not a massive job, it's been done and it worked, the problem is that locking all the CPython refcount updates slowed down the important single-cpu case so it was never adopted. So the real massive job would be moving to a tracing GC and changing every extension module ever written. But, that's what PyPy is (among other things) so I'm waiting to hear from Laura why PyPy has a GIL. From no.email at nospam.invalid Mon Feb 23 20:50:24 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 17:50:24 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> Message-ID: <871tlgaxi7.fsf@jester.gateway.pace.com> Chris Angelico writes: >> What if you want to dynamically construct a callable and send it to >> another process? > I'm not sure what that would actually mean. Do you try to construct it > out of code that already exists in the other process? Are you passing > actual code to the other process? I gave an example in a reply to Steven, something like other_thread_queue.put(lambda x: x*x) to tell the other thread it is supposed to square something. It receives a callable and calls it in its own context. From rosuav at gmail.com Mon Feb 23 21:03:06 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 13:03:06 +1100 Subject: Future of Pypy? In-Reply-To: <871tlgaxi7.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <871tlgaxi7.fsf@jester.gateway.pace.com> Message-ID: On Tue, Feb 24, 2015 at 12:50 PM, Paul Rubin wrote: > Chris Angelico writes: >>> What if you want to dynamically construct a callable and send it to >>> another process? >> I'm not sure what that would actually mean. Do you try to construct it >> out of code that already exists in the other process? Are you passing >> actual code to the other process? > > I gave an example in a reply to Steven, something like > > other_thread_queue.put(lambda x: x*x) > > to tell the other thread it is supposed to square something. It > receives a callable and calls it in its own context. So, you would have to pass code to the other process, probably. What about this: y = 4 other_thread_queue.put(lambda x: x*y) Or this: y = [4] def next_y(): y[0] += 1 return y[0] other_thread_queue.put(next_y) It may not be obvious with your squaring example, but every Python function has its context (module globals, etc). You can't pass a function around without also passing, or sharing, its data. With threads in a single process, this isn't a problem. They all access the same memory space, so they can all share state. As soon as you go to separate processes, these considerations become serious. ChrisA From steve+comp.lang.python at pearwood.info Mon Feb 23 21:07:52 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 13:07:52 +1100 Subject: Python shell: Arrow keys not working in PuTTY References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> Message-ID: <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> Laura Creighton wrote: > DO NOT REBUILD PYTHON ON CENTOS! > > It can break the whole package management system > which depends on having a particular version of python installed. > > If you are running Centos you need to use virtualenv to be safe. > > Laura Almost right! You can install Python from source. Unzip the source tar ball, cd into the source directory, and run: ./configure make BUT do *not* run `make install` as that will overwrite your system Python and Bad Things will happen. Instead, run `make altinstall`. Now you will have two Python installs, the stock-standard system Python is available as "python", and the newly-installed one as "pythonX.Y" (for version X.Y). What I do is then add alias python /usr/local/bin/pythonX.Y to my .bashrc file, and use the new one. (Don't forget to change the X.Y to whatever version you actually used.) What if you accidentally blow away the system Python by running `make install`? You probably won't be able to use yum to fix it, since yum relies on the system Python. Fortunately, you haven't actually *removed* the system Python, it's still there just hiding. First, find out which version of Python your system Python is: [steve at ando ~]$ ls -l /usr/bin/python* -rwxr-xr-x 2 root root 5708 Jan 9 2013 /usr/bin/python lrwxrwxrwx 1 root root 6 Jan 22 2013 /usr/bin/python2 -> python -rwxr-xr-x 2 root root 5708 Jan 9 2013 /usr/bin/python2.4 This tells me that the system Python is 2.4 (well actually I already knew that). /usr/bin/python here is a hard-link to the python2.4 executable. If I had accidentally killed the system Python, I would see something like: -rwxr-xr-x 2 root root 6789 Feb 24 2015 /usr/bin/python lrwxrwxrwx 1 root root 6 Jan 22 2013 /usr/bin/python2 -> python -rwxr-xr-x 1 root root 5708 Jan 9 2013 /usr/bin/python2.4 -rwxr-xr-x 2 root root 6789 Feb 24 2015 /usr/bin/python2.7 The /usr/bin/python is now hard-linked to the newly installed version. As root: rm /usr/bin/python # Die, imposter! ln /usr/bin/python2.4 /usr/bin/python # Revive the system Python. All fixed! -- Steven From steve+comp.lang.python at pearwood.info Mon Feb 23 21:15:46 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 13:15:46 +1100 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ebded4$0$11116$c3e8da3@news.astraweb.com> Ethan Furman wrote: > On 02/22/2015 11:41 PM, Steven D'Aprano wrote: > >> If you want *CPython* to work without a GIL, well, are you volunteering >> to do the work? It is a massive job, and the core devs aren't terribly >> interested. Probably because they understand that the GIL is not often an >> unsurmountable problem in real life. That brings us to the point I was >> making: I believe, and the core devs appear to think the same, that the >> *actual* number of people who would benefit from CPython losing the GIL >> is not large enough to justify the work it would require. > > If memory serves, the first and primary point to losing the GIL is that > single-threaded code must run just as fast (or at least close to just as > fast) as the GIL version, and previous attempts have all badly failed that > requisite. That was then. Now even smartphones have multiple cores. It might be time to reconsider. If not now, then surely by the time entry-level machines have 8 cores, the requirement that 1-core machines aren't penalized will surely be dropped :-) It may turn out that removing the GIL doesn't improve things *enough*, who knows what will happen. I recall from the first attempt, back in Python 1.5 days, that there was a 40% slowdown for single core machines, a very slight speedup for two-core, and increasing the number of cores beyond four made no real difference. So it might turn out that "removing the GIL" is great in theory and not so useful in practice, but I guess only time will tell. -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 23 21:24:07 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 13:24:07 +1100 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> Message-ID: <54ebe0c8$0$11114$c3e8da3@news.astraweb.com> Ben Finney wrote: >> In C language, there is &A for address of A > > There is no ?address of a value? concept in Python. You access a value > by some reference, either a name or an item in a collection. When a > reference changes to reference some different value, other references > are not affected. Furthermore, in Python, values can *change their address*. Jython and IronPython both are built on top of a garbage collector which can move memory around, meaning that objects can end up in a completely different location. This makes no difference to Python code, and there is no supported way of getting to the address of an object. (In fact, objects may be split over multiple locations, even in CPython, e.g. the new optimized implementation for dicts has shared storage for keys.) -- Steve From nad at acm.org Mon Feb 23 21:35:39 2015 From: nad at acm.org (Ned Deily) Date: Mon, 23 Feb 2015 18:35:39 -0800 Subject: Python shell: Arrow keys not working in PuTTY References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> Message-ID: In article <54ebdcfa$0$11100$c3e8da3 at news.astraweb.com>, Steven D'Aprano wrote: > Almost right! > > You can install Python from source. Unzip the source tar ball, cd into the > source directory, and run: > > ./configure > make > > BUT do *not* run `make install` as that will overwrite your system Python > and Bad Things will happen. Instead, run `make altinstall`. With no --prefix= on ./configure, the default install location is to /usr/local, so "make install" would install a link at /usr/local/bin/python (or python3) and it would only overwrite your system Python if the system Python happened to be installed in /usr/local/bin/. -- Ned Deily, nad at acm.org From ijrosario at students.pccc.edu Mon Feb 23 22:56:38 2015 From: ijrosario at students.pccc.edu (Iris J Rosario) Date: Tue, 24 Feb 2015 03:56:38 +0000 Subject: Compatibility Issues - Windows 8 Message-ID: Hello, I've been having problems with running python IDLE on my Windows 8 tower. Can you please let me know where I can rea up on how to correct it? Or which version will work best on my computer? Best Regards, Iris J Rosario ijrosario at students.pccc.edu "One can complain because rosebushes have thorns, or rejoice because thorn bushes have roses." ~ Abraham Lincoln -------------- next part -------------- An HTML attachment was scrubbed... URL: From no.email at nospam.invalid Mon Feb 23 23:40:40 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 20:40:40 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <871tlgaxi7.fsf@jester.gateway.pace.com> Message-ID: <87mw43apmf.fsf@jester.gateway.pace.com> Chris Angelico writes: > So, you would have to pass code to the other process, probably. What > about this: > y = 4 > other_thread_queue.put(lambda x: x*y) the y in the lambda is a free variable that's a reference to the surrounding mutable context, so that's at best dubious. You could use: other_thread_queue.put(lambda x, y=y: x*y) > Or this: > > y = [4] > def next_y(): > y[0] += 1 > return y[0] > other_thread_queue.put(next_y) There you have shared mutable data, which isn't allowed in this style. > It may not be obvious with your squaring example, but every Python > function has its context (module globals, etc). You can't pass a > function around without also passing, or sharing, its data. That is ok as long as the data can't change. > With threads in a single process, this isn't a problem. They all > access the same memory space, so they can all share state. As soon as > you go to separate processes, these considerations become serious. Right, that's a limitation of processes compared to threads. From tjreedy at udel.edu Mon Feb 23 23:53:14 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Feb 2015 23:53:14 -0500 Subject: Compatibility Issues - Windows 8 In-Reply-To: References: Message-ID: On 2/23/2015 10:56 PM, Iris J Rosario wrote: > Hello, > > I?ve been having problems with running python IDLE on my Windows 8 > tower. Can you please let me know where I can rea up on how to correct > it? Or which version will work best on my computer? How did you install Python? How are you trying to run Idle? -- Terry Jan Reedy From marcos.al.azevedo at gmail.com Mon Feb 23 23:59:43 2015 From: marcos.al.azevedo at gmail.com (Marcos Almeida Azevedo) Date: Tue, 24 Feb 2015 12:59:43 +0800 Subject: Compatibility Issues - Windows 8 In-Reply-To: References: Message-ID: Hi, Iris Did you try installing via msi installer? Example https://www.python.org/ftp/python/3.4.3/python-3.4.3rc1.msi On Tue, Feb 24, 2015 at 12:53 PM, Terry Reedy wrote: > On 2/23/2015 10:56 PM, Iris J Rosario wrote: > >> Hello, >> >> I?ve been having problems with running python IDLE on my Windows 8 >> tower. Can you please let me know where I can rea up on how to correct >> it? Or which version will work best on my computer? >> > > How did you install Python? How are you trying to run Idle? > > > > -- > Terry Jan Reedy > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Tue Feb 24 00:20:29 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Feb 2015 18:20:29 +1300 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: Cem Karan wrote: > I tend to structure my code as a tree or DAG of objects. The owner refers to > the owned object, but the owned object has no reference to its owner. With > callbacks, you get cycles, where the owned owns the owner. This is why I suggested registering a listener object plus a method name instead of a callback. It avoids that reference cycle, because there is no long-lived callback object keeping a reference to the listener. -- Greg From no.email at nospam.invalid Tue Feb 24 00:27:42 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 21:27:42 -0800 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> Message-ID: <87lhjnang1.fsf@jester.gateway.pace.com> Ryan Stuart writes: > I'm not sure what else to say really. It's just a fact of life that > Threads by definition run in the same memory space and hence always > have the possibility of nasty unforeseen problems. They are unforeseen > because it is extremely difficult (maybe impossible?) to try and map > out and understand all the different possible mutations to state. Sure, the shared memory introduces the possibility of some bad errors, I'm just saying that I've found that by staying with a certain straightforward style, it doesn't seem difficult in practice to avoid those errors. > Sure, your code might not be making any mutations (that you know of), > but malloc definitely is [1], and that's just the tip of the iceberg. > Other things like buffers for stdin and stdout, DNS resolution etc. > all have the same issue. I don't understand what you mean about malloc. I looked at that code and there's a mutex to make multi-threaded programs work right, and an ifdef (maybe for better performance) to use different code if there are no threads. IOW they spent a bunch of time handling threads. Are you saying there's a bug? Re stdin/stdout: obviously you can't have multiple threads messing with the same fd's; that's the same thing as data sharing. Re DNS: if gethostbyname isn't thread-safe I'd think of that as a pretty bad bug. But I'm having a vague memory of having had an issue with this though, and IIRC it took part of a morning to figure out what was going on, annoying but not a multi-month bug-hunt or anything like that. It didn't happen on my workstation, but only on the embedded target that was probably running an old or weird libc. > To borrow from the original article I linked - "Nevertheless I still > think it?s a bad idea to make things harder for ourselves if we can > avoid it." That article was interesting in some ways but confused in others. One way it was interesting is it said various non-thread approaches (such as coroutines) had about the same problems as threads. Some ways it was confused were: 1) thinking Haskell threads were like processes with separate address spaces. In fact they are in the same address space and programming with them isn't all that different from Python threads, though the synchronization primitives are a bit different. There is also an STM library available that is ingenious though apparently somewhat slow. 2) it has a weird story about the brass cockroach, that basically signified that they didn't have a robust enough testing system to be able to reproduce the bug. That is what they should have worked on. 3) It goes into various hazards of the balance transfer example not mentioning that STM (available in Haskell and Clojure) completely solves it. 4) It says: "eventually a system which communicates exclusively through non-blocking queues effectively becomes a set of communicating event loops, and its problems revert to those of an event-driven system; it doesn?t look like regular programming with threads any more." That is essentially what an Erlang program is, and it misses the fact that those low-level event loops can use blocking operations to their heart's content, without the inversion of control (callback spaghetti) of traditional evented systems (I haven't used asyncio yet). Also, the low-level loops can run in parallel on multiple cores, while a asyncio-style coroutine loop is sequential under the skin. In Erlang/OTP, you don't even see the event loops directly, since they are abstracted away by the OTP framework and it looks like RPC calls at the application level. But, it helps to know what is going on underneath. I'm realizing some people program Python in an ultra-dynamic style where the mutability of modules, functions, etc. really comes into play, so that make threads much more dangerous. I've tended to write Python with much less dynamism or even as if it were statically typed, so maybe that helps. Anyway, I got one thing out of this, which is that the multiprocessing module looks pretty nice and I should try it even when I don't need multicore parallelism, so thanks for that. In reality though, Python is still my most productive language for throwaway, non-concurrent scripting, but for more complicated concurrent programs, alternatives like Haskell, Erlang, and Go all have significant attractions. From random832 at fastmail.us Tue Feb 24 00:29:51 2015 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 24 Feb 2015 00:29:51 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <1424755791.3675849.231568073.1541B308@webmail.messagingengine.com> On Tue, Feb 24, 2015, at 00:20, Gregory Ewing wrote: > Cem Karan wrote: > > I tend to structure my code as a tree or DAG of objects. The owner refers to > > the owned object, but the owned object has no reference to its owner. With > > callbacks, you get cycles, where the owned owns the owner. > > This is why I suggested registering a listener object > plus a method name instead of a callback. It avoids that > reference cycle, because there is no long-lived callback > object keeping a reference to the listener. How does that help? Everywhere you would have had a reference to the "callback object", you now have a reference to the listener object. You're just shuffling deck chairs around: if B shouldn't reference A because A owns B, then removing C from the B->C->A reference chain does nothing to fix this. From greg.ewing at canterbury.ac.nz Tue Feb 24 00:45:18 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Feb 2015 18:45:18 +1300 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: Cem Karan wrote: > On Feb 22, 2015, at 5:15 AM, Gregory Ewing > wrote: > >> Perhaps instead of registering a callback function, you should be >> registering the listener object together with a method name. > > I see what you're saying, but I don't think it gains us too much. If I store > an object and an unbound method of the object, or if I store the bound method > directly, I suspect it will yield approximately the same results. It would be weird and unpythonic to have to register both an object and an unbound method, and if you use a bound method you can't keep a weak reference to it. -- Greg From rosuav at gmail.com Tue Feb 24 00:57:00 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 16:57:00 +1100 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: <87lhjnang1.fsf@jester.gateway.pace.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> Message-ID: On Tue, Feb 24, 2015 at 4:27 PM, Paul Rubin wrote: >> Sure, your code might not be making any mutations (that you know of), >> but malloc definitely is [1], and that's just the tip of the iceberg. >> Other things like buffers for stdin and stdout, DNS resolution etc. >> all have the same issue. > > Re stdin/stdout: obviously you can't have > multiple threads messing with the same fd's; that's the same thing as > data sharing. Actually, you can quite happily have multiple threads messing with the underlying file descriptors, that's not a problem. (Though you will tend to get interleaved output. But if you always produce output in single blocks of text that each contain one line with a trailing newline, you should see interleaved lines that are each individually correct. I'm also not sure of any sane way to multiplex stdin - merging output from multiple threads is fine, but dividing input between multiple threads is messy.) The problem is *buffers* for stdin and stdout, where you have to be absolutely sure that you're not trampling all over another thread's data structures. If you unbuffer your output, it's probably going to be thread-safe. ChrisA From steve+comp.lang.python at pearwood.info Tue Feb 24 00:59:59 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 16:59:59 +1100 Subject: Python shell: Arrow keys not working in PuTTY References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> Message-ID: <54ec1360$0$12978$c3e8da3$5496439d@news.astraweb.com> Ned Deily wrote: > In article <54ebdcfa$0$11100$c3e8da3 at news.astraweb.com>, > Steven D'Aprano wrote: >> Almost right! >> >> You can install Python from source. Unzip the source tar ball, cd into >> the source directory, and run: >> >> ./configure >> make >> >> BUT do *not* run `make install` as that will overwrite your system Python >> and Bad Things will happen. Instead, run `make altinstall`. > > With no --prefix= on ./configure, the default install location is to > /usr/local, so "make install" would install a link at > /usr/local/bin/python (or python3) and it would only overwrite your > system Python if the system Python happened to be installed in > /usr/local/bin/. Well, I'm not going to say you are wrong, but I can say that on Centos 5 systems (and presumably that includes Fedora and Red Hat of the equivalent vintage), if you just run `make install` it overwrites the /usr/bin/python hard link to /usr/bin/python2.4. I know because I've done it :-( -- Steve From no.email at nospam.invalid Tue Feb 24 01:23:46 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 23 Feb 2015 22:23:46 -0800 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> Message-ID: <87h9ubakul.fsf@jester.gateway.pace.com> Chris Angelico writes: > Actually, you can quite happily have multiple threads messing with the > underlying file descriptors,... The problem is *buffers* for stdin > and stdout, where you have to be absolutely sure that you're not > trampling all over another thread's data structures. Oh ok, sure, yeah, the distinction is valid. I guess the classic interleaved "print 'a'" "print 'b'" loops could crash if the stdio structures get corrupted through conflicting updates somehow. From steve+comp.lang.python at pearwood.info Tue Feb 24 01:56:03 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 17:56:03 +1100 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ec2085$0$11103$c3e8da3@news.astraweb.com> Chris Angelico wrote: > On Mon, Feb 23, 2015 at 6:41 PM, Steven D'Aprano > wrote: >>>> - and you must be using libraries and tools which prevent you moving to >>>> Jython or IronPython or some other alternative. >>> >>> I don't get this at all. Why should I not want Python to have the same >>> capabilities? >> >> Python does have the same capabilities. Jython and IronPython aren't >> different languages, they are Python. > > Presuming that he meant CPython, one other good reason for wanting to > stick with it rather than jump to another interpreter is all the new > features of the recent versions. (Do Jython or IronPython support > Python 3.x at all, even?) Every new release of CPython has new and > cool features, and the other Pythons are generally playing catch-up. Most people are not using the bleeding edge version of Python, and even those who do, aren't usually using it in production. There are still plenty of people using Python 2.3 in production, and even a few using 1.5. But as you say, there are good reasons for wishing to stick to CPython over Jython, IronPython, PyPy or Stackless, let alone less mature or experimental implementations. I get that. But the point I am trying to make is that there are an awful lot of people who mindlessly bash "Python" the language for "the GIL". The argument is even worse than "Python sux cos significant whitespace", because the whitespace issue at least is mostly a matter for personal preference, but the GIL appears to be a technical, objective judgement. Trouble is that it is *wrong*. Python the language doesn't have a GIL. Implementations with a GIL aren't slower because of it, they are faster. And removing the GIL doesn't necessarily speed up multithreaded code. So they are technically wrong, and in practical terms often wrong too. I am *happy* when people come up with nuanced and considered opinions that they found that Python was unsuitable for some specific project, and can give good cogent reasons for it. If they can demonstrate reasons for believing that CPython would have been suitable *if the GIL was removed*, that's great. Given sufficiently many such projects, and 1- and 2-core CPUs gradually becoming obsolete, the time may come for somebody else to take a shot at removing the GIL from CPython again. That time might even be now, if there is a volunteer. And C coders out there want a nice meaty project to work on during rainy weekends? -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 24 01:57:19 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 24 Feb 2015 17:57:19 +1100 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <871tlgaxi7.fsf@jester.gateway.pace.com> <87mw43apmf.fsf@jester.gateway.pace.com> Message-ID: <54ec20d0$0$11103$c3e8da3@news.astraweb.com> Paul Rubin wrote: >> With threads in a single process, this isn't a problem. They all >> access the same memory space, so they can all share state. As soon as >> you go to separate processes, these considerations become serious. > > Right, that's a limitation of processes compared to threads. > I think the point is that it's not a *limitation* of processes, but a *feature* of processes that they don't share state. (Well, I think there are explicit ways to have shared memory, but that's another story.) An interesting point of view: threading is harmful because it removes determinism from your program. http://radar.oreilly.com/2007/01/threads-considered-harmful.html As I once wrote: A programmer had a problem, and thought Now he has "I know, I'll solve two it with threads!" problems. http://code.activestate.com/lists/python-list/634273/ Some discussion of the pros and cons of threading: http://c2.com/cgi/wiki?ThreadsConsideredHarmful -- Steven From rosuav at gmail.com Tue Feb 24 02:16:34 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 18:16:34 +1100 Subject: Future of Pypy? In-Reply-To: <54ec2085$0$11103$c3e8da3@news.astraweb.com> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> <54ec2085$0$11103$c3e8da3@news.astraweb.com> Message-ID: On Tue, Feb 24, 2015 at 5:56 PM, Steven D'Aprano wrote: > Most people are not using the bleeding edge version of Python, and even > those who do, aren't usually using it in production. There are still plenty > of people using Python 2.3 in production, and even a few using 1.5. > > But as you say, there are good reasons for wishing to stick to CPython over > Jython, IronPython, PyPy or Stackless, let alone less mature or experimental > implementations. I get that. Yes, not everyone is running Python 3.5 in production, I totally get that :) But if I'm writing an app to run on Debian or Ubuntu, I can depend on there being a CPython 3.x available, even on the oldest currently-supported releases (Ubuntu Lucid and Debian Squeeze both ship Python 3.1), and it won't be long before saying "requires Python 3.3" won't be a problem. Plus, you can always compile CPython from source on any Unix-like system, or download the .msi for Windows, and run 3.4 with no problems. But if you want to use PyPy, Jython, or any other implementation of the language, you're quite possibly stuck on 2.x, or if 3.x support does exist, it's the new and experimental code. Even a willingness to compile from source won't get you past that, so using any of the new features of Python 3.x is likely to cut out some of the alternate interpreters. It's fine to use Python 2.3 if that's all you need (and if you don't need any bug fixes from python.org, which presumably means you have security support from someone else). But it's also fine to want to use a newer Python. It'd be nice if all the major Pythons supported 3.4, but the reality of volunteer time is that CPython is pretty much always going to be in the lead. ChrisA From wxjmfauth at gmail.com Tue Feb 24 02:57:43 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Mon, 23 Feb 2015 23:57:43 -0800 (PST) Subject: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> <54ec2085$0$11103$c3e8da3@news.astraweb.com> Message-ID: <1fd652d5-f58f-4bd7-bced-af60adc304d9@googlegroups.com> Blah blah blah... pypy fails as soon as one lives the ascii world. (Not exactly for the same reasons as in CPy 3.3+.) Regards. jmf From marko at pacujo.net Tue Feb 24 03:08:18 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 24 Feb 2015 10:08:18 +0200 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> Message-ID: <87bnkjenpp.fsf@elektro.pacujo.net> Chris Angelico : > Actually, you can quite happily have multiple threads messing with the > underlying file descriptors, that's not a problem. (Though you will > tend to get interleaved output. But if you always produce output in > single blocks of text that each contain one line with a trailing > newline, you should see interleaved lines that are each individually > correct. I'm also not sure of any sane way to multiplex stdin - > merging output from multiple threads is fine, but dividing input > between multiple threads is messy.) The problem is *buffers* for stdin > and stdout, where you have to be absolutely sure that you're not > trampling all over another thread's data structures. If you unbuffer > your output, it's probably going to be thread-safe. Here's an anecdote describing one real-life threading problem. We had a largish multithreading framework (in Java, but I'm setting it in Python and in a much simplified form). We were mindful of deadlocks caused by lock reversal so we had come up with a policy whereby objects form a layered hierarchy. An object higher up in the hierarchy was allowed to call methods of objects below while holding locks. The opposite was not allowed; if an object desired to call a method of an object above it (through a registered callback), it had to relinquish all locks before doing so. However, a situation like this arose: class App: def send_stream(self, sock): with self.lock: self.register_socket(sock) class SocketWrapper: def read(_, count): return sock.recv(count) def close(_): sock.close() with self.lock: self.unregister_socket(sock) self.transport.forward_and_close(SocketWrapper(sock)) class Transport: def forward_and_close(self, readable): with self.lock: more = readable.read(1000) if more is WOULDBLOCK: self.reschedule(readable) elif more: ... # out of scope for the anecdote else: # EOF reached readable.close() Now the dreaded lock reversal arises when the App object calls self.transport.forward_and_close() and Transport calls readable.close() at the same time. So why lock categorically like that? Java has a handy "synchronized" keyword that wraps the whole method in "with self.lock". Ideally, that handy idiom could be employed methodically. More importantly, to avoid locking problems, the methodology should be rigorous and mindless. If the developer must perform a deep locking analysis at every turn, they are bound to make mistakes, especially when more than one developer is involved, with differing intuitions. Unfortunately, that deep locking analysis *is* required at every turn, and mistakes *are* bound to happen. Marko From marko at pacujo.net Tue Feb 24 03:12:12 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 24 Feb 2015 10:12:12 +0200 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <54ead9aa$0$12930$c3e8da3$5496439d@news.astraweb.com> <8761asaxmz.fsf@jester.gateway.pace.com> Message-ID: <877fv7enj7.fsf@elektro.pacujo.net> Paul Rubin : > Deadlocks and livelocks can happen in multi-process code just like > with threads. The main thing to avoid is to assume that writing can block until its completion. At least one of the communicating parties must write in a nonblocking manner and make sure it is always ready to read. The remaining problem is flow control to prevent write buffer flooding. All of that is easier to get right than the intricacies of multithreading. Marko From marko at pacujo.net Tue Feb 24 03:18:22 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 24 Feb 2015 10:18:22 +0200 Subject: list storing variables References: <54eb2357$0$3011$426a74cc@news.free.fr> <54ebe0c8$0$11114$c3e8da3@news.astraweb.com> Message-ID: <87385ven8x.fsf@elektro.pacujo.net> Steven D'Aprano : > Ben Finney wrote: >> There is no ?address of a value? concept in Python. > > Furthermore, in Python, values can *change their address*. > > Jython and IronPython both are built on top of a garbage collector > which can move memory around, meaning that objects can end up in a > completely different location. This makes no difference to Python > code, and there is no supported way of getting to the address of an > object. (In fact, objects may be split over multiple locations, even > in CPython, e.g. the new optimized implementation for dicts has shared > storage for keys.) You are taking the concept of "address" far too literally, and yet not literally enough. Even in C programs, the *physical* address of a variable/value can change at any point without warning. In abstract terms, an address is a reference. And that reference stays still in both C and Python. What happens under the hood of the data model is irrelevant. Marko From jldunn2000 at gmail.com Tue Feb 24 03:37:44 2015 From: jldunn2000 at gmail.com (loial) Date: Tue, 24 Feb 2015 00:37:44 -0800 (PST) Subject: Concatenate list values In-Reply-To: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> Message-ID: <3ee01d6c-3a79-4f25-b425-0ce5b4b178f1@googlegroups.com> Many thanks for those you chose to help me out. Problem solved. From wxjmfauth at gmail.com Tue Feb 24 05:30:45 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 24 Feb 2015 02:30:45 -0800 (PST) Subject: Concatenate list values In-Reply-To: References: <12821378-62af-4954-8b61-aa0738c5ff5c@googlegroups.com> <20150223101434.52871fcc@bigbox.christie.dr> Message-ID: Le lundi 23 f?vrier 2015 18:15:11 UTC+1, Mark Lawrence a ?crit?: > On 23/02/2015 16:14, Tim Chase wrote: > > On 2015-02-23 07:58, loial wrote: > >> Is there a quick way to concatenate all the values in a list into a > >> string, except the first value? > >> > >> I want this to work with variable length lists. > >> > >> All values in list will be strings. > > > > Using > > > > "".join(my_list[1:]) > > > > should work. > > > > If it is an arbitrary iterable instead of a list (and thus can't be > > sliced), you can use > > > > "".join(itertools.islice(my_iter, 1, None)) > > > > -tkc > > > > Presumably rather more efficient for long lists as you're not taking a > copy of (effectively) the entire list. Any volunteers to measure it, > not mentioning any names, Mr D'Aprano? :) > > -- >>> timeit.repeat("'a'.join(['a']*123)") [2.747081951123846, 2.7309830094075096, 2.7286731458373197] >>> timeit.repeat("'\u1234'.join(['a']*123)") [2.7187153298930298, 2.7063901499382155, 2.709908310428318] >>> >>> timeit.repeat("'a'.join(['a']*123)") [2.935619986680422, 2.931890334845548, 2.9307190587571768] >>> timeit.repeat("'\u1234'.join(['a']*123)") [6.233187127305428, 6.218316962117797, 6.228552434296944] >>> What is wrong by design will always stays wrong. (On top of this, it's buggy) jmf From David.Aldrich at EMEA.NEC.COM Tue Feb 24 05:47:11 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Tue, 24 Feb 2015 10:47:11 +0000 Subject: How best to install path.py package on Centos 5? In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B945390@EX10MBX02.EU.NEC.COM> References: <41302A7145AC054FA7A96CFD03835A0A0B945390@EX10MBX02.EU.NEC.COM> Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B945861@EX10MBX02.EU.NEC.COM> I fixed this by using pip instead of easy_install: pip3.4 install path.py -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrick.brihaye at gmail.com Tue Feb 24 05:49:18 2015 From: pierrick.brihaye at gmail.com (pierrick.brihaye at gmail.com) Date: Tue, 24 Feb 2015 02:49:18 -0800 (PST) Subject: Newbie question about text encoding Message-ID: Hello, Working with pyshp, this is my code : import shapefile inFile = shapefile.Reader("blah") for sr in inFile.shapeRecords(): rec = sr.record[2] print("Output : ", rec, type(rec)) Output: hippodrome du resto Output: b'stade de man\xe9 braz' Why do I get 2 different types ? How to get a string object when I have accented characters ? Thank you, p.b. From cfkaran2 at gmail.com Tue Feb 24 06:00:55 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 24 Feb 2015 06:00:55 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <39813568-6DB8-4341-A130-C256CFF352EE@gmail.com> <54e8c017$0$13008$c3e8da3$5496439d@news.astraweb.com> <51947083-01AC-4D1E-ACBD-3C476446B527@gmail.com> <54EA43C8.2090309@stoneleaf.us> <87wq39hbq1.fsf@elektro.pacujo.net><0443FBDF-9FD0-492E-8486-03EBD0972C04@gmail.com> <201502222229.t1MMT7Dn029607@fido.openend.se> Message-ID: On Feb 23, 2015, at 7:29 AM, "Frank Millman" wrote: > > "Cem Karan" wrote in message > news:A3C11A70-5846-4915-BB26-B23793B65670 at gmail.com... >> >> >> Good questions! That was why I was asking about 'gotchas' with WeakSets >> originally. Honestly, the only way to know for sure would be to write two >> APIs for doing similar things, and then see how people react to them. The >> problem is, how do you set up such a study so it is statistically valid? >> > > Just in case you missed Steven's comment on my 'gotcha', and my reply, it is > worth repeating that what I reported as a gotcha was not what it seemed. > > If you set up the callback as a weakref, and the listening object goes out > of scope, it will wait to be garbage collected. However, as far as I can > tell, the weakref is removed at the same time as the object is gc'd, so > there is no 'window' where the weakref exists but the object it is > referencing does not exist. > > My problem was that I had performed a cleanup operation on the listening > object before letting it go out of scope, and it was no longer in a valid > state to deal with the callback, resulting in an error. If you do not have > that situation, your original idea may well work. Thank you Frank, I did read Steve's comment to your reply earlier, but what you said in your original reply made sense to me. I don't have control over user code. That means that if someone wants to write code such that they perform some kind of cleanup and are no longer able to handle the callback, they are free to do so. While I can't prevent this from happening, I can make it as obvious as possible in my code that before you perform any cleanup, you also need to unregister from the library. That is my main goal in developing pythonic/obvious methods of registering callbacks. Thanks, Cem Karan From cfkaran2 at gmail.com Tue Feb 24 06:06:33 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 24 Feb 2015 06:06:33 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <54e8af1b$0$12976$c3e8da3$5496439d@news.astraweb.com> Message-ID: I'm combining two messages into one, On Feb 24, 2015, at 12:29 AM, random832 at fastmail.us wrote: > On Tue, Feb 24, 2015, at 00:20, Gregory Ewing wrote: >> Cem Karan wrote: >>> I tend to structure my code as a tree or DAG of objects. The owner refers to >>> the owned object, but the owned object has no reference to its owner. With >>> callbacks, you get cycles, where the owned owns the owner. >> >> This is why I suggested registering a listener object >> plus a method name instead of a callback. It avoids that >> reference cycle, because there is no long-lived callback >> object keeping a reference to the listener. > > How does that help? Everywhere you would have had a reference to the > "callback object", you now have a reference to the listener object. > You're just shuffling deck chairs around: if B shouldn't reference A > because A owns B, then removing C from the B->C->A reference chain does > nothing to fix this. On Feb 24, 2015, at 12:45 AM, Gregory Ewing wrote: > Cem Karan wrote: >> On Feb 22, 2015, at 5:15 AM, Gregory Ewing >> wrote: >>> Perhaps instead of registering a callback function, you should be >>> registering the listener object together with a method name. >> I see what you're saying, but I don't think it gains us too much. If I store >> an object and an unbound method of the object, or if I store the bound method >> directly, I suspect it will yield approximately the same results. > > It would be weird and unpythonic to have to register both > an object and an unbound method, and if you use a bound > method you can't keep a weak reference to it. Greg, random832 said what I was thinking earlier, that you've only increased the diameter of your cycle without actually fixing it. Can you give a code example where your method breaks the cycle entirely? Thanks, Cem Karan From rosuav at gmail.com Tue Feb 24 06:09:30 2015 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Feb 2015 22:09:30 +1100 Subject: Newbie question about text encoding In-Reply-To: References: Message-ID: On Tue, Feb 24, 2015 at 9:49 PM, wrote: > Working with pyshp, this is my code : > > import shapefile > > inFile = shapefile.Reader("blah") > > for sr in inFile.shapeRecords(): > rec = sr.record[2] > print("Output : ", rec, type(rec)) > > Output: hippodrome du resto > Output: b'stade de man\xe9 braz' > > Why do I get 2 different types ? > How to get a string object when I have accented characters ? I don't know what pyshp is doing here, so you may want to seek a pyshp-specific mailing list for help. My guess is that it's automatically decoding to str if it's ASCII-only, and giving you back the raw bytes if there are any that it can't handle. The question is: What encoding _is_ that? Do you know what character you're expecting to see there? Before you can turn that into a string, you have to figure out whether it's Latin-1 (ISO-8859-1), or some other ISO-8859-x standard, or a Windows codepage, or an ancient thing off a Mac, or whatever else it might be. Once you know that, it's easy: you just decode() the bytes objects. But you MUST figure out the encoding first. ChrisA From David.Aldrich at EMEA.NEC.COM Tue Feb 24 06:18:38 2015 From: David.Aldrich at EMEA.NEC.COM (David Aldrich) Date: Tue, 24 Feb 2015 11:18:38 +0000 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <54ec1360$0$12978$c3e8da3$5496439d@news.astraweb.com> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> <54ec1360$0$12978$c3e8da3$5496439d@news.astraweb.com> Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B9458C4@EX10MBX02.EU.NEC.COM> > >> BUT do *not* run `make install` as that will overwrite your system > >> Python and Bad Things will happen. Instead, run `make altinstall`. Thanks for all the warnings. We did use `make altinstall`, so all is ok. Recompiling, with readline installed, fixed the arrow keys. From davea at davea.name Tue Feb 24 06:25:24 2015 From: davea at davea.name (Dave Angel) Date: Tue, 24 Feb 2015 06:25:24 -0500 Subject: Newbie question about text encoding In-Reply-To: References: Message-ID: <54EC5FA4.6070703@davea.name> On 02/24/2015 05:49 AM, pierrick.brihaye at gmail.com wrote: > Hello, > > Working with pyshp, this is my code : What version of Python, what version of pyshp, from where, and what OS? These are the first information to supply in any query that goes outside of the standard library. For example, you might be running CPython 3.2.1 on Ubuntu 14.04.1, and installed pyshp 1.2.1 from https://pypi.python.org/pypi/pyshp Or some other combination. > > import shapefile > > inFile = shapefile.Reader("blah") > > for sr in inFile.shapeRecords(): > rec = sr.record[2] > print("Output : ", rec, type(rec)) > > Output: hippodrome du resto > Output: b'stade de man\xe9 braz' > > Why do I get 2 different types ? > How to get a string object when I have accented characters ? > > Thank you, > > p.b. > From my (cursory) reading of the pyshp docs on the above page, I cannot see what the [2] element of the record list should look like. So I'd have to guess. The bytes object is presumably an encoded version of the character string. I don't see anything on that page about unicode, or decode, so you might have to guess the encoding. Anyway, you can decode the bytestring into a regular string if you can correctly guess the encoding method, such as utf-8. If that were the right decoding, you could just use mystring = rec.decode() But utf-8 does not seem to be the right encoding for that bytestring. So you'll need a form like: mystring = rec.decode(encoding='xxx') for some value of xxx. -- DaveA From mail at timgolden.me.uk Tue Feb 24 06:49:06 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 24 Feb 2015 11:49:06 +0000 Subject: Python Silent Install In-Reply-To: <91A935D75D3AF940BAD7318B212C554B019D5DDB2F@mb-01.uni.ad.sunderland.ac.uk> References: <91A935D75D3AF940BAD7318B212C554B019D5DD4D3@mb-01.uni.ad.sunderland.ac.uk> <54EB4748.6010601@timgolden.me.uk> <54EB5462.70104@timgolden.me.uk> <91A935D75D3AF940BAD7318B212C554B019D5DDB2F@mb-01.uni.ad.sunderland.ac.uk> Message-ID: <54EC6532.70700@timgolden.me.uk> On 23/02/2015 16:41, Colin Atkinson wrote: > Fantastic! I ran the below command and it worked: > > Msiexec /i python.msi TARGETDIR="C:\Program Files\Python" ADDLOCAL="all" REMOVE="pip_feature" /qn > > Thanks for the quick response. Thanks for confirming that the answer worked. One thing to note: this incantation doesn't just remove the aesthetic nuisance of a "DOS Box" appearing. It actually prevents pip from being installed. This isn't necessarily a problem; you might not want it installed, and there are several ways of getting it there after the fact. But you need to be aware that this is the effect. TJG From fabiofz at gmail.com Tue Feb 24 08:23:13 2015 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Tue, 24 Feb 2015 10:23:13 -0300 Subject: Design thought for callbacks In-Reply-To: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: Hi Cem, I didn't read the whole long thread, but I thought I'd point you to what I'm using in PyVmMonitor (http://www.pyvmmonitor.com/) -- which may already cover your use-case. Take a look at the callback.py at https://github.com/fabioz/pyvmmonitor-core/blob/master/pyvmmonitor_core/callback.py And its related test (where you can see how to use it): https://github.com/fabioz/pyvmmonitor-core/blob/master/_pyvmmonitor_core_tests/test_callback.py (note that it falls back to a strong reference on simple functions -- i.e.: usually top-level methods or methods created inside a scope -- but otherwise uses weak references). Best Regards, Fabio On Sat, Feb 21, 2015 at 12:44 AM, Cem Karan wrote: > Hi all, I'm working on a project that will involve the use of callbacks, > and I want to bounce an idea I had off of everyone to make sure I'm not > developing a bad idea. Note that this is for python 3.4 code; I don't need > to worry about any version of python earlier than that. > > In order to inform users that certain bits of state have changed, I > require them to register a callback with my code. The problem is that when > I store these callbacks, it naturally creates a strong reference to the > objects, which means that if they are deleted without unregistering > themselves first, my code will keep the callbacks alive. Since this could > lead to really weird and nasty situations, I would like to store all the > callbacks in a WeakSet ( > https://docs.python.org/3/library/weakref.html#weakref.WeakSet). That > way, my code isn't the reason why the objects are kept alive, and if they > are no longer alive, they are automatically removed from the WeakSet, > preventing me from accidentally calling them when they are dead. My > question is simple; is this a good design? If not, why not? Are there any > potential 'gotchas' I should be worried about? > > Thanks, > Cem Karan > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lac at openend.se Tue Feb 24 09:51:17 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 15:51:17 +0100 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: Message from David Aldrich of "Tue, 24 Feb 2015 11:18:38 +0000." <41302A7145AC054FA7A96CFD03835A0A0B9458C4@EX10MBX02.EU.NEC.COM> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> <54ec1360$0$12978$c3e8da3$5496439d@news.astraweb.com><41302A7145AC054FA7A96CFD03835A0A0B9458C4@EX10MBX02.EU.NEC.COM> Message-ID: <201502241451.t1OEpHaD015528@fido.openend.se> In a message of Tue, 24 Feb 2015 11:18:38 +0000, David Aldrich writes: >> >> BUT do *not* run `make install` as that will overwrite your system >> >> Python and Bad Things will happen. Instead, run `make altinstall`. > >Thanks for all the warnings. We did use `make altinstall`, so all is ok. > >Recompiling, with readline installed, fixed the arrow keys. Great! Thank you for letting us know. Laura (who was worried) From lac at openend.se Tue Feb 24 09:55:41 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 15:55:41 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Dave Angel of "Tue, 24 Feb 2015 06:25:24 -0500." <54EC5FA4.6070703@davea.name> References: <54EC5FA4.6070703@davea.name> Message-ID: <201502241455.t1OEtffT016452@fido.openend.se> In a message of Tue, 24 Feb 2015 06:25:24 -0500, Dave Angel writes: >But utf-8 does not seem to be the right encoding for that bytestring. >So you'll need a form like: > mystring = rec.decode(encoding='xxx') > >for some value of xxx. >DaveA And the xxx you want is "latin1" Laura From rosuav at gmail.com Tue Feb 24 10:03:16 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 02:03:16 +1100 Subject: Newbie question about text encoding In-Reply-To: <201502241455.t1OEtffT016452@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> Message-ID: On Wed, Feb 25, 2015 at 1:55 AM, Laura Creighton wrote: > In a message of Tue, 24 Feb 2015 06:25:24 -0500, Dave Angel writes: >>But utf-8 does not seem to be the right encoding for that bytestring. >>So you'll need a form like: >> mystring = rec.decode(encoding='xxx') >> >>for some value of xxx. > >>DaveA > > And the xxx you want is "latin1" Can you be sure it's Latin-1? I'm not certain of that. In any case, I never advocate fixing encoding problems by "just do this and it'll all go away"; you have to understand your data before you can decode it. ChrisA From lac at openend.se Tue Feb 24 10:06:19 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 16:06:19 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Laura Creighton of "Tue, 24 Feb 2015 15:55:41 +0100." <201502241455.t1OEtffT016452@fido.openend.se> References: <54EC5FA4.6070703@davea.name><201502241455.t1OEtffT016452@fido.openend.se> Message-ID: <201502241506.t1OF6Jav018601@fido.openend.se> In a message of Tue, 24 Feb 2015 15:55:41 +0100, Laura Creighton writes: >In a message of Tue, 24 Feb 2015 06:25:24 -0500, Dave Angel writes: >>But utf-8 does not seem to be the right encoding for that bytestring. >>So you'll need a form like: >> mystring = rec.decode(encoding='xxx') >> >>for some value of xxx. > >>DaveA > >And the xxx you want is "latin1" > >Laura er, latin1. You don't want an extra set of quotes. There are many aliases for latin1. i.e. latin_1, iso-8859-1, iso8859-1, 8859, cp819, latin, latin1, L1 see: https://docs.python.org/2.4/lib/standard-encodings.html and you might want to read https://docs.python.org/2/howto/unicode.html to understand the problem better. Laura From lac at openend.se Tue Feb 24 10:07:36 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 16:07:36 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Chris Angelico of "Wed, 25 Feb 2015 02:03:16 +1100." References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> Message-ID: <201502241507.t1OF7aUm018883@fido.openend.se> In a message of Wed, 25 Feb 2015 02:03:16 +1100, Chris Angelico writes: >On Wed, Feb 25, 2015 at 1:55 AM, Laura Creighton wrote: >> In a message of Tue, 24 Feb 2015 06:25:24 -0500, Dave Angel writes: >>>But utf-8 does not seem to be the right encoding for that bytestring. >>>So you'll need a form like: >>> mystring = rec.decode(encoding='xxx') >>> >>>for some value of xxx. >> >>>DaveA >> >> And the xxx you want is "latin1" > >Can you be sure it's Latin-1? I'm not certain of that. In any case, I >never advocate fixing encoding problems by "just do this and it'll all >go away"; you have to understand your data before you can decode it. > >ChrisA I can, I speak French and I recognise the data. It's French place names, places where sporting events are held. :) Laura From rosuav at gmail.com Tue Feb 24 10:10:42 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 02:10:42 +1100 Subject: Newbie question about text encoding In-Reply-To: <201502241507.t1OF7aUm018883@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> Message-ID: On Wed, Feb 25, 2015 at 2:07 AM, Laura Creighton wrote: >>Can you be sure it's Latin-1? I'm not certain of that. In any case, I >>never advocate fixing encoding problems by "just do this and it'll all >>go away"; you have to understand your data before you can decode it. >> >>ChrisA > > I can, I speak French and I recognise the data. It's French place names, > places where sporting events are held. :) Ah, okay. :) But even with that level of confidence, you still have to pick between Latin-1 and CP-1252, which you can't tell based on this one snippet. Welcome to untagged encodings. ChrisA From lac at openend.se Tue Feb 24 10:24:00 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 16:24:00 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Chris Angelico of "Wed, 25 Feb 2015 02:10:42 +1100." References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> Message-ID: <201502241524.t1OFO09k022270@fido.openend.se> In a message of Wed, 25 Feb 2015 02:10:42 +1100, Chris Angelico writes: >On Wed, Feb 25, 2015 at 2:07 AM, Laura Creighton wrote: >>>Can you be sure it's Latin-1? I'm not certain of that. In any case, I >>>never advocate fixing encoding problems by "just do this and it'll all >>>go away"; you have to understand your data before you can decode it. >>> >>>ChrisA >> >> I can, I speak French and I recognise the data. It's French place names, >> places where sporting events are held. :) > >Ah, okay. :) But even with that level of confidence, you still have to >pick between Latin-1 and CP-1252, which you can't tell based on this >one snippet. Welcome to untagged encodings. > >ChrisA Ah, yes, you are right about that. I see CP-1252 about 2 times every 10 years, and latin1 every minute of my life, so I am biased to assume I know what I am seeing. ChrisA, you come from an English speaking country, right? For those of us who come from countries whose language doesn't fit in ASCII, the notion of 'understand the data' doesn't work very well. We already understand the data -- its a set of words in our native language. The hard part isn't understanding the data, but rather understanding how the hell Python could be so stupid as to not understand it. :) The notion that Python normally only understands the subset of the characters in your native language than English speakers use in their language is not the most obvious thing. And having taught countless European kids how to write their very first program in Python, I can tell you for certain that the sort of deep understanding of encoding methods is not what 10 year olds who just want to print out the names of their friends, and their favourite music titles, and their favourite musicians want to know. :) Laura From rosuav at gmail.com Tue Feb 24 10:33:30 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 02:33:30 +1100 Subject: Newbie question about text encoding In-Reply-To: <201502241524.t1OFO09k022270@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> Message-ID: On Wed, Feb 25, 2015 at 2:24 AM, Laura Creighton wrote: > Ah, yes, you are right about that. I see CP-1252 about 2 times every 10 > years, and latin1 every minute of my life, so I am biased to assume I > know what I am seeing. Fair enough. CP-1252 is still a possibility, but the difference can be dealt with later. > ChrisA, you come from an English speaking country, right? Yes (Australia, to be specific). > For those of us who come from countries whose language doesn't fit in > ASCII, the notion of 'understand the data' doesn't work very well. We > already understand the data -- its a set of words in our native language. > The hard part isn't understanding the data, but rather understanding how > the hell Python could be so stupid as to not understand it. :) The > notion that Python normally only understands the subset of the > characters in your native language than English speakers use in their > language is not the most obvious thing. Also a reasonable baseline assumption; but the trouble is that if you automatically assume that text is encoded in your favourite eight-bit system, you're taking a huge risk. Now, you have a huge leg up on me, in that you actually recognize the *words* in that piece of text. That means you can have MUCH greater confidence in stating that it's Latin-1 than I can. But that's precisely what I mean by "understand the data". If you, being a native French speaker, pick up a file written in (say) Polish, and encoded Latin-2, you'll recognize by the ASCII characters that it's not French text, and probably you'd be able to spot that it ought to be Latin-2 rather than Latin-1. That's understanding the data, that's having more information than just the byte patterns. A computer can't reliably do that (just look up the "Bush hid the facts" bug if you don't believe me), but a human often can. > And having taught countless European kids how to write their very first > program in Python, I can tell you for certain that the sort of deep > understanding of encoding methods is not what 10 year olds who just > want to print out the names of their friends, and their favourite > music titles, and their favourite musicians want to know. :) Right, so you should be teaching them to use Python 3, and always saving everything in UTF-8, and basically ignoring the whole mess of eight-bit encodings :) ChrisA From random832 at fastmail.us Tue Feb 24 10:38:23 2015 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 24 Feb 2015 10:38:23 -0500 Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> Message-ID: <1424792303.3362888.231766513.193BCCE7@webmail.messagingengine.com> On Tue, Feb 24, 2015, at 10:10, Chris Angelico wrote: > Ah, okay. :) But even with that level of confidence, you still have to > pick between Latin-1 and CP-1252, which you can't tell based on this > one snippet. Welcome to untagged encodings. Or Latin-9 (ISO 8859-15) That was popular on Linux systems for a while before everyone switched to UTF-8 - it's got the Euro sign, and (relevant to French) the "oe" ligature, and uppercase Y with diaeresis, at the expense of "generic currency" and fractions. Or it could be Latin-3, Latin-5 (8859-9), or Latin-8 (8859-14) - they are not commonly used for French locales, being primarily intended for other languages, but they do support all characters (at least all from Latin-1) used in French names. I assume there are likewise several Windows codepages it could be. From wxjmfauth at gmail.com Tue Feb 24 11:01:55 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Tue, 24 Feb 2015 08:01:55 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name><201502241455.t1OEtffT016452@fido.openend.se> Message-ID: <7f2bdec0-483a-4ffb-ad64-09a687f385ba@googlegroups.com> Sorry, you are all wrong. The coding is UCS1. Python is so funny. jmf From lac at openend.se Tue Feb 24 11:20:41 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 17:20:41 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Chris Angelico of "Wed, 25 Feb 2015 02:33:30 +1100." References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> Message-ID: <201502241620.t1OGKf4n002146@fido.openend.se> In a message of Wed, 25 Feb 2015 02:33:30 +1100, Chris Angelico writes: >Also a reasonable baseline assumption; but the trouble is that if you >automatically assume that text is encoded in your favourite eight-bit >system, you're taking a huge risk. But, you know, I wasn't assuming this. I actually read latin1. I could read it in ascii, know that \xe9 means '?', a letter combination that we have in Swedish, so I am rather used to reading, and then well, I could read all of his strings, know they were in French, and know that latin1 was what he needed things to be decoded to. >Now, you have a huge leg up on me, in that you actually recognize the >*words* in that piece of text. That means you can have MUCH greater >confidence in stating that it's Latin-1 than I can. But that's >precisely what I mean by "understand the data". If you, being a native >French speaker, pick up a file written in (say) Polish, and encoded >Latin-2, you'll recognize by the ASCII characters that it's not French >text, and probably you'd be able to spot that it ought to be Latin-2 >rather than Latin-1. That's understanding the data, that's having more >information than just the byte patterns. A computer can't reliably do >that (just look up the "Bush hid the facts" bug if you don't believe >me), but a human often can. Absolutely correct. But you must not require that all of the speakers of non-English languages think about their languages as 'special encodings'. Only the monoglot ever think of a foreign language as a code. That poor guy the original poster just wants to have a nice string of his sporting event place name. We should tell him how to get that, not how to be an expert in all the encodings on the face of this earth. Chances are, the only thing he needs to talk about are French words. If not, well, he will come back when things stop working, and have lots more data to give him. If, instead, this makes him go away happy, then this was the very best thing to do. >> And having taught countless European kids how to write their very first >> program in Python, I can tell you for certain that the sort of deep >> understanding of encoding methods is not what 10 year olds who just >> want to print out the names of their friends, and their favourite >> music titles, and their favourite musicians want to know. :) > >Right, so you should be teaching them to use Python 3, and always >saving everything in UTF-8, and basically ignoring the whole mess of >eight-bit encodings :) Of course this makes sense. But you seem to be missing the point. People who are asking for help in getting things to work in their native language need a 'do this quick' sort of answer. The deeper problems of supporting all languages and language encodings can very much wait. The OP wants a hunk of bytes that happens to mean something in French, and is not encodable in the limited English language to work like a different hunk of bytes that means something in French but is encodable. Don't overburden them. >ChrisA Laura From rosuav at gmail.com Tue Feb 24 11:24:32 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 03:24:32 +1100 Subject: Newbie question about text encoding In-Reply-To: <201502241620.t1OGKf4n002146@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> Message-ID: On Wed, Feb 25, 2015 at 3:20 AM, Laura Creighton wrote: > People who are asking for help in getting things to work in their > native language need a 'do this quick' sort of answer. The deeper > problems of supporting all languages and language encodings can very > much wait. I'm not so sure about that. When "supporting all languages" is as simple as "use Python 3 and UTF-8 everywhere", the cost is much lower than it might be, and the benefit is potentially huge. A "do this quick" answer might get you by *right now*, but it leaves open the possibility of subtler errors. That's why Python moved to Unicode-by-default, even though eight-bit encodings will tend to produce the right results for simple text. ChrisA From johnnybureddu at gmx.com Tue Feb 24 12:05:46 2015 From: johnnybureddu at gmx.com (REVOLUCION!) Date: Tue, 24 Feb 2015 09:05:46 -0800 (PST) Subject: -- redacted -- Message-ID: <8c78eea0-9184-4e40-a16e-d6df129fb1a8@googlegroups.com> -- redacted -- From dieffedue at gmx.com Tue Feb 24 12:11:25 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. DECALIA GENEVE.) Date: Tue, 24 Feb 2015 09:11:25 -0800 (PST) Subject: -- redacted -- In-Reply-To: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> References: <27a7e94b-8df2-420b-b83f-75629ae225fe@googlegroups.com> Message-ID: -- redacted -- From dieffedue at gmx.com Tue Feb 24 12:11:54 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. DECALIA GENEVE.) Date: Tue, 24 Feb 2015 09:11:54 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From davea at davea.name Tue Feb 24 12:13:24 2015 From: davea at davea.name (Dave Angel) Date: Tue, 24 Feb 2015 12:13:24 -0500 Subject: Newbie question about text encoding In-Reply-To: <201502241620.t1OGKf4n002146@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> Message-ID: <54ECB134.5090304@davea.name> On 02/24/2015 11:20 AM, Laura Creighton wrote: > In a message of Wed, 25 Feb 2015 02:33:30 +1100, Chris Angelico writes: >> Also a reasonable baseline assumption; but the trouble is that if you >> automatically assume that text is encoded in your favourite eight-bit >> system, you're taking a huge risk. > > But, you know, I wasn't assuming this. I actually read latin1. I > could read it in ascii, know that \xe9 means '?', a letter combination > that we have in Swedish, so I am rather used to reading, and then > well, I could read all of his strings, know they were in French, > and know that latin1 was what he needed things to be decoded to. With a sample of one string, how did you read "all his strings". And with one non-ASCII code in that single string, how did you know that 'latin1' was the only encoding that included a reasonable character at that encoding? > >> Now, you have a huge leg up on me, in that you actually recognize the >> *words* in that piece of text. That means you can have MUCH greater >> confidence in stating that it's Latin-1 than I can. But that's >> precisely what I mean by "understand the data". If you, being a native >> French speaker, pick up a file written in (say) Polish, and encoded >> Latin-2, you'll recognize by the ASCII characters that it's not French >> text, and probably you'd be able to spot that it ought to be Latin-2 >> rather than Latin-1. That's understanding the data, that's having more >> information than just the byte patterns. A computer can't reliably do >> that (just look up the "Bush hid the facts" bug if you don't believe >> me), but a human often can. > > Absolutely correct. But you must not require that all of the speakers > of non-English languages think about their languages as 'special > encodings'. Only the monoglot ever think of a foreign language as > a code. All languages are foreign. All that can be written to a disk file are bytes. Those have to have been encoded to represent some abstraction called a character set, or string. The question is whether the encoding method is specified for the particular file type, or for the particular file. See http://support.esri.com/cn/knowledgebase/techarticles/detail/21106 according to that page, starting at ArcGIS 10.2.1, the default sets the code page to UTF-8 (UNICODE) in the shapefile (.DBF) But in earlier ones, there's supposed to be a reference to the codepage used. From that, one can presumably derive which decoder to use. > > That poor guy the original poster just wants to have a nice string > of his sporting event place name. We should tell him how to get that, > not how to be an expert in all the encodings on the face of this earth. > Chances are, the only thing he needs to talk about are French words. > > If not, well, he will come back when things stop working, and have lots > more data to give him. If, instead, this makes him go away happy, then > this was the very best thing to do. > >>> And having taught countless European kids how to write their very first >>> program in Python, I can tell you for certain that the sort of deep >>> understanding of encoding methods is not what 10 year olds who just >>> want to print out the names of their friends, and their favourite >>> music titles, and their favourite musicians want to know. :) >> >> Right, so you should be teaching them to use Python 3, and always >> saving everything in UTF-8, and basically ignoring the whole mess of >> eight-bit encodings :) > > Of course this makes sense. But you seem to be missing the point. > People who are asking for help in getting things to work in their > native language need a 'do this quick' sort of answer. The deeper > problems of supporting all languages and language encodings can very > much wait. The OP wants a hunk of bytes that happens to mean > something in French, and is not encodable in the limited English > language to work like a different hunk of bytes that means something > in French but is encodable. > > Don't overburden them. My guess is that this is only appropriate for users who use only locally created data. Since the OP's data is apparently old (if it were current versions, it'd have been utf-8), who knows how consistent the encoding is. -- -- DaveA From dieffedue at gmx.com Tue Feb 24 12:14:39 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. DECALIA GENEVE.) Date: Tue, 24 Feb 2015 09:14:39 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: -- redacted -- From dieffedue at gmx.com Tue Feb 24 12:19:08 2015 From: dieffedue at gmx.com (ALFREDO PIACENTINI. DECALIA GENEVE.) Date: Tue, 24 Feb 2015 09:19:08 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: <68f91a40-933f-4c27-866f-ef432dd92fc5@googlegroups.com> Message-ID: -- redacted -- From torriem at gmail.com Tue Feb 24 12:39:16 2015 From: torriem at gmail.com (Michael Torrie) Date: Tue, 24 Feb 2015 10:39:16 -0700 Subject: Python shell: Arrow keys not working in PuTTY In-Reply-To: <201502231727.t1NHRCQO013874@fido.openend.se> References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <201502231727.t1NHRCQO013874@fido.openend.se> Message-ID: <54ECB744.1020004@gmail.com> On 02/23/2015 10:27 AM, Laura Creighton wrote: > And it is time for me to get on a train. Can somebody please, please > make sure the OP doesn't shoot himself in the foot? > > Laura It's okay for now; he's trying to use Python 3. The system Python in CentOS is Python 2, so he can mess with 3 all he would like (for now). From invalid at invalid.invalid Tue Feb 24 12:49:42 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 24 Feb 2015 17:49:42 +0000 (UTC) Subject: How to apply python patch for issue9729 References: Message-ID: On 2015-02-24, Chris Angelico wrote: > On Tue, Feb 24, 2015 at 8:09 AM, Sada Shirol wrote: >> Upon some research found that I need to apply below patch: >> >> http://bugs.python.org/issue9729 >> >> How do I apply a patch to python 2.6.6 on Linux? > > Hmm. You have a bit of a problem there: The patch wasn't designed to > apply to 2.6. So you may find that it doesn't apply at all, or needs > some tweaking. Or worse yet, it applies just fine but doesn't work quite right in particular cases that you won't run across until later when it's much more embarassing. -- Grant Edwards grant.b.edwards Yow! ... I want to perform at cranial activities with gmail.com Tuesday Weld!! From emile at fenx.com Tue Feb 24 12:57:06 2015 From: emile at fenx.com (Emile van Sebille) Date: Tue, 24 Feb 2015 09:57:06 -0800 Subject: Future of Pypy? In-Reply-To: <201502221722.t1MHM16H001195@fido.openend.se> References: of "Sun, 22 Feb 2015 12:45:03 +0000." <201502221722.t1MHM16H001195@fido.openend.se> Message-ID: On 2/22/2015 9:22 AM, Laura Creighton wrote: > There was, and still is, an enormous amount of resentment about this. > For a lot of people, the perception was, and still is that the benefits > of Python 3.x over Python 2.x was not worth breaking backwards > compatibility. And there still are plenty of places whose plan is > to use Python 2.7 indefinitely into the far future. I've got 15 > years worth of commercial python code out there in the world, and > nobody wants it converted enough to pay me to do so. Their position > is that it runs quite well enough as it is. I'm sure not > going to convert the stuff for fun. Practically every Python consultant on > the planet is in the same boat. +1 As a glue language, it's simply impractical to reapply the glue. Emile From lac at openend.se Tue Feb 24 14:45:54 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 20:45:54 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Dave Angel of "Tue, 24 Feb 2015 12:13:24 -0500." <54ECB134.5090304@davea.name> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name> Message-ID: <201502241945.t1OJjshO013092@fido.openend.se> In a message of Tue, 24 Feb 2015 12:13:24 -0500, Dave Angel writes: >With a sample of one string, how did you read "all his strings". And >with one non-ASCII code in that single string, how did you know that >'latin1' was the only encoding that included a reasonable character at >that encoding? Ah, 2 strings. And I did not promise that latin1 was the only encoding that included a reasonable char at his encoding. I only proinmised that it was one that did. And, given the nature of the data, I was pretty sure that this was the one he wanted. If it did not work, he would come back and complain. >See http://support.esri.com/cn/knowledgebase/techarticles/detail/21106 > >according to that page, starting at ArcGIS 10.2.1, the default sets the >code page to UTF-8 (UNICODE) in the shapefile (.DBF) Who cares. In Europe, among Europeans, we are used to seeing Latin1 or Latin2. >My guess is that this is only appropriate for users who use only locally >created data. Since the OP's data is apparently old (if it were current >versions, it'd have been utf-8), who knows how consistent the encoding is. I do. Very much so. The idea that the whole world loves utf-8 is nonsense. Most of europe has been using latin1, latin2 etc. before unicode was invented and will, as far as I know, continue to use it. Oldness is an indication that latin1 is more likely to be the encoding than uft-8. Your guess is that latin1 is only used in local encodings. My data is that, we in Western Europe, have this format pretty much all of the time, for everywhere, unless you are only doing local encodings (in which case you would use utf-8) Laura From lac at openend.se Tue Feb 24 14:57:53 2015 From: lac at openend.se (Laura Creighton) Date: Tue, 24 Feb 2015 20:57:53 +0100 Subject: Newbie question about text encoding In-Reply-To: Message from Laura Creighton of "Tue, 24 Feb 2015 20:45:54 +0100." <201502241945.t1OJjshO013092@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name><201502241945.t1OJjshO013092@fido.openend.se> Message-ID: <201502241957.t1OJvrJS015604@fido.openend.se> Dave Angel are you another Native English speaker living in a world where ASCII is enough? Laura From blakemalc66 at gmail.com Tue Feb 24 15:13:24 2015 From: blakemalc66 at gmail.com (blakemalc66 at gmail.com) Date: Tue, 24 Feb 2015 12:13:24 -0800 (PST) Subject: Compatibility Issues - Windows 8 In-Reply-To: References: Message-ID: <0bbecc0a-9e03-4441-9fa8-c96bd3f690ec@googlegroups.com> On Tuesday, February 24, 2015 at 9:36:20 PM UTC+11, Marcos Almeida Azevedo wrote: > Hi, Iris > > Did you try installing via msi installer? Example https://www.python.org/ftp/python/3.4.3/python-3.4.3rc1.msi > > > > On Tue, Feb 24, 2015 at 12:53 PM, Terry Reedy wrote: > On 2/23/2015 10:56 PM, Iris J Rosario wrote: > > > Hello, > > > > I've been having problems with running python IDLE on my Windows 8 > > tower. Can you please let me know where I can rea up on how to correct > > it? Or which version will work best on my computer? > > > > > How did you install Python? How are you trying to run Idle? > > > > > > > > -- > > Terry Jan Reedy > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > -- > > > > > > > Marcos | I love PHP, Linux, and Java Hi all I too can not get idle to run on win 8.1 using python3.4.2 installed from the python-3.4.2.amd64.msi. pythonw.exe appears in the task manager but no gui is displayed. Cheers Malcolm From davea at davea.name Tue Feb 24 15:41:44 2015 From: davea at davea.name (Dave Angel) Date: Tue, 24 Feb 2015 15:41:44 -0500 Subject: Newbie question about text encoding In-Reply-To: <201502241957.t1OJvrJS015604@fido.openend.se> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name><201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <54ECE208.3050903@davea.name> On 02/24/2015 02:57 PM, Laura Creighton wrote: > Dave Angel > are you another Native English speaker living in a world where ASCII > is enough? I'm a native English speaker, and 7 bits is not nearly enough. Even if I didn't currently care, I have some history: No. CDC display code is enough. Who needs lowercase? No. Baudot code is enough. No, EBCDIC is good enough. Who cares about other companies. No, the "golf-ball" only holds this many characters. If we need more, we can just get the operator to switch balls in the middle of printing. No. 2 digit years is enough. This world won't last till the millennium anyway. No. 2k is all the EPROM you can have. Your code HAS to fit in it, and only 1.5k RAM. No. 640k is more than anyone could need. No, you cannot use a punch card made on a model 26 keypunch in the same deck as one made on a model 29. Too bad, many of the codes are different. (This one cost me travel back and forth between two different locations with different model keypunches) No. 8 bits is as much as we could ever use for characters. Who could possibly need names or locations outside of this region? Or from multiple places within it? 35 years ago I helped design a serial terminal that "spoke" Chinese, using a two-byte encoding. But a single worldwide standard didn't come until much later, and I cheered Unicode when it was finally unveiled. I've worked with many printers that could only print 70 or 80 unique characters. The laser printer, and even the matrix printer are relatively recent inventions. Getting back on topic: According to: http://support.esri.com/cn/knowledgebase/techarticles/detail/27345 """ArcGIS Desktop applications, such as ArcMap, are Unicode based, so they support Unicode to a certain level. The level of Unicode support depends on the data format.""" That page was written about 2004, so there was concern even then. And according to another, """In the header of each shapefile (.DBF), a reference to a code page is included.""" -- DaveA From greg.ewing at canterbury.ac.nz Tue Feb 24 16:19:18 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 25 Feb 2015 10:19:18 +1300 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: random832 at fastmail.us wrote: > On Tue, Feb 24, 2015, at 00:20, Gregory Ewing wrote: > >>This is why I suggested registering a listener object >>plus a method name instead of a callback. It avoids that >>reference cycle, because there is no long-lived callback >>object keeping a reference to the listener. > > How does that help? Everywhere you would have had a reference to the > "callback object", you now have a reference to the listener object. The point is that the library can keep a weak reference to the listener object, whereas it can't reliably keep a weak reference to a bound method. -- Greg From roy at panix.com Tue Feb 24 16:34:42 2015 From: roy at panix.com (Roy Smith) Date: Tue, 24 Feb 2015 13:34:42 -0800 (PST) Subject: Bug in timsort!? Message-ID: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ From zachary.ware+pylist at gmail.com Tue Feb 24 16:40:42 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Tue, 24 Feb 2015 15:40:42 -0600 Subject: Bug in timsort!? In-Reply-To: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Tue, Feb 24, 2015 at 3:34 PM, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ http://bugs.python.org/issue23515 Note that the article does mention that Python is not vulnerable due to this bug (and best I can tell has no behavioral issues); no computer currently in existence can create a large enough array to cause a problem. -- Zach From breamoreboy at yahoo.co.uk Tue Feb 24 16:49:30 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 24 Feb 2015 21:49:30 +0000 Subject: Bug in timsort!? In-Reply-To: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 24/02/2015 21:34, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > As you can clearly no longer rely on Python it looks like, after a long and loving relationship, I'm forced into moving on. PHP, Applescript, Javascript or back to C or even CORAL 66, what is your (plural) recommendation(s)? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marko at pacujo.net Tue Feb 24 17:21:32 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 25 Feb 2015 00:21:32 +0200 Subject: Newbie question about text encoding References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> Message-ID: <87ioerc5n7.fsf@elektro.pacujo.net> Laura Creighton : > Who cares. In Europe, among Europeans, we are used to seeing > Latin1 or Latin2. No, it's UCS-2 (Windows) or UTF-8 (Linux) -- among us Europeans. > The idea that the whole world loves utf-8 is nonsense. Windows people don't care for UTF-8, they don't have to. Linux people use it. Love is not necessary. Me, I use en_US.UTF-8. > Most of europe has been using latin1, latin2 etc. before unicode was > invented and will, as far as I know, continue to use it. Oldness is an > indication that latin1 is more likely to be the encoding than uft-8. Latin-1 is confined to HTML, if even there. > My data is that, we in Western Europe, have this format pretty much > all of the time, for everywhere, unless you are only doing local > encodings (in which case you would use utf-8) There's a third way, but it's not in Western Europe, as far as I can tell. Japan is another story. I don't know about Russia. Marko From sohcahtoa82 at gmail.com Tue Feb 24 17:36:13 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Tue, 24 Feb 2015 14:36:13 -0800 (PST) Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Tuesday, February 24, 2015 at 1:50:15 PM UTC-8, Mark Lawrence wrote: > On 24/02/2015 21:34, Roy Smith wrote: > > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > > > > As you can clearly no longer rely on Python it looks like, after a long > and loving relationship, I'm forced into moving on. PHP, Applescript, > Javascript or back to C or even CORAL 66, what is your (plural) > recommendation(s)? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence Personally, I'm going with BASIC being interpreted by a JavaScript script being fed from a PHP script running on an out-dated version of Apache. It includes an in-browser IDE that will be written as an ActiveX control and will require Internet Explorer 6. An Adobe Flash version is in the works to allow it to run on modern browsers. From invalid at invalid.invalid Tue Feb 24 17:45:58 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Tue, 24 Feb 2015 22:45:58 +0000 (UTC) Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 2015-02-24, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ I don't get it. 3.2 Corrected Python merge_collapse function merge_collapse(MergeState *ms) { struct s_slice *p = ms->pending; assert(ms); while (ms->n > 1) { Py_ssize_t n = ms->n - 2; if ( n > 0 && p[n-1].len <= p[n].len + p[n+1].len || (n-1 > 0 && p[n-2].len <= p[n].len + p[n-1].len)) { if (p[n-1].len < p[n+1].len) --n; if (merge_at(ms, n) < 0) return -1; } else if (p[n].len <= p[n+1].len) { if (merge_at(ms, n) < 0) return -1; } else break; } return 0; } Or does "Python function" mean something else in this context? -- Grant Edwards grant.b.edwards Yow! I request a weekend in at Havana with Phil Silvers! gmail.com From ian.g.kelly at gmail.com Tue Feb 24 17:59:34 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 24 Feb 2015 15:59:34 -0700 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Tue, Feb 24, 2015 at 3:45 PM, Grant Edwards wrote: > On 2015-02-24, Roy Smith wrote: > >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > > I don't get it. > > 3.2 Corrected Python merge_collapse function > > merge_collapse(MergeState *ms) > { > struct s_slice *p = ms->pending; > > assert(ms); > while (ms->n > 1) { > Py_ssize_t n = ms->n - 2; > if ( n > 0 && p[n-1].len <= p[n].len + p[n+1].len > || (n-1 > 0 && p[n-2].len <= p[n].len + p[n-1].len)) { > if (p[n-1].len < p[n+1].len) > --n; > if (merge_at(ms, n) < 0) > return -1; > } > else if (p[n].len <= p[n+1].len) { > if (merge_at(ms, n) < 0) > return -1; > } > else > break; > } > return 0; > } > > Or does "Python function" mean something else in this context? Recall that CPython is implemented in C. ;-) From fomcl at yahoo.com Tue Feb 24 18:05:14 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Tue, 24 Feb 2015 15:05:14 -0800 Subject: Python Silent Install In-Reply-To: <54EC6532.70700@timgolden.me.uk> Message-ID: <1424819114.89841.YahooMailMobile@web163806.mail.gq1.yahoo.com> Personally I find that Python is incomplete without pip and setuptools. -------------- next part -------------- An HTML attachment was scrubbed... URL: From baykiwi at gmail.com Tue Feb 24 18:05:22 2015 From: baykiwi at gmail.com (baykiwi at gmail.com) Date: Tue, 24 Feb 2015 15:05:22 -0800 (PST) Subject: strip bug? Message-ID: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> >>> 'http://xthunder'.strip('http://') 'xthunder' >>> 'http://thunder'.strip('http://') 'under' >>> I could understand backslash but forward slash? From xcesar at gmail.com Tue Feb 24 18:15:54 2015 From: xcesar at gmail.com (Eduardo) Date: Tue, 24 Feb 2015 15:15:54 -0800 (PST) Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: <411cc0ce-3df3-4e33-8853-60838c1c292b@googlegroups.com> Well, from the docstring of strip: ------ S.strip([chars]) -> string or unicode Return a copy of the string S with leading and trailing whitespace removed. If chars is given and not None, remove characters in chars instead. If chars is unicode, S will be converted to unicode before stripping ------ In this case 'http://thunder'.strip('http://') is the same as 'http://thunder'.strip('htp:/') and any character of the string that is passed to the method will be removed. From ian.g.kelly at gmail.com Tue Feb 24 18:16:57 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 24 Feb 2015 16:16:57 -0700 Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: On Tue, Feb 24, 2015 at 4:05 PM, wrote: >>>> 'http://xthunder'.strip('http://') > 'xthunder' >>>> 'http://thunder'.strip('http://') > 'under' >>>> This removes all leading and trailing occurrences of the characters in the string 'http://', not the exact substring 'http://'. For that, use either the str.replace method or slicing. > I could understand backslash but forward slash? I don't understand the question. From irmen.NOSPAM at xs4all.nl Tue Feb 24 18:19:00 2015 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Wed, 25 Feb 2015 00:19:00 +0100 Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: <54ed06e1$0$2875$e4fe514c@news.xs4all.nl> On 25-2-2015 0:05, baykiwi at gmail.com wrote: >>>> 'http://xthunder'.strip('http://') > 'xthunder' >>>> 'http://thunder'.strip('http://') > 'under' >>>> > > I could understand backslash but forward slash? > >>> help("".strip) Help on built-in function strip: strip(...) method of builtins.str instance S.strip([chars]) -> str Return a copy of the string S with leading and trailing whitespace removed. If chars is given and not None, remove characters in chars instead. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It strips any chars given in the argument, not as a prefix/suffix to remove. Irmen From ckaynor at zindagigames.com Tue Feb 24 18:22:06 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Tue, 24 Feb 2015 15:22:06 -0800 Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: On Tue, Feb 24, 2015 at 3:05 PM, wrote: > >>> 'http://xthunder'.strip('http://') > 'xthunder' > >>> 'http://thunder'.strip('http://') > 'under' > >>> > > I could understand backslash but forward slash? I believe the issue is that str.strip does not do quite what you are thinking it does, however your message is very unspecific about what you expect to get. It seems that you except str.strip to remove a sub-string, while it actually removes the list of specified characters (see https://docs.python.org/3.4/library/stdtypes.html?highlight=str.strip#str.strip). As such, you would get the same result from 'http://thunder'.strip('htp:/') and 'http://thunder'.strip('/thp:') as from your samples. To get what I am guessing you want ("xthunder" for the first and "thunder" for the second), you should use splicing, likely combined with str.startswith() (untested code): url = 'http://thunder/ if url.startswith('http://'): url = url[7:] else # Handle case without the prefix in whatever manner is correct for your case. Possibly continue, possibly error out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfgang.maier at biologie.uni-freiburg.de Tue Feb 24 18:25:27 2015 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 25 Feb 2015 00:25:27 +0100 Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: On 25.02.2015 00:05, baykiwi at gmail.com wrote: >>>> 'http://xthunder'.strip('http://') > 'xthunder' >>>> 'http://thunder'.strip('http://') > 'under' >>>> > > I could understand backslash but forward slash? > You are misunderstanding what the argument to strip does. Look at this: >>> 'http://xthunder'.strip('/hpt:') 'xthunder' From rosuav at gmail.com Tue Feb 24 18:33:03 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 10:33:03 +1100 Subject: Bug in timsort!? In-Reply-To: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, Feb 25, 2015 at 8:34 AM, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ The post links to: http://svn.python.org/projects/python/trunk/Objects/listobject.c Is that still valid and current? It says svn, but does it pop through to the Mercurial repo where the _real_ CPython code is stored? ChrisA From python at mrabarnett.plus.com Tue Feb 24 18:38:38 2015 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 24 Feb 2015 23:38:38 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: <54ED0B7E.7030801@mrabarnett.plus.com> On 2015-02-24 21:40, Zachary Ware wrote: > On Tue, Feb 24, 2015 at 3:34 PM, Roy Smith wrote: >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > > http://bugs.python.org/issue23515 > > Note that the article does mention that Python is not vulnerable due > to this bug (and best I can tell has no behavioral issues); no > computer currently in existence can create a large enough array to > cause a problem. > I think the key word here is "currently". From skip.montanaro at gmail.com Tue Feb 24 18:50:41 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 24 Feb 2015 17:50:41 -0600 Subject: Bug in timsort!? In-Reply-To: <54ED0B7E.7030801@mrabarnett.plus.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: On Tue, Feb 24, 2015 at 5:38 PM, MRAB wrote: > I think the key word here is "currently". Sure, but it's not like you have to put out security updates tomorrow. Even if/when we get to the point where machines can hold an array of 2**49 elements, I suspect people won't be using straight Python to wrangle them. They will likely use other languages, and if using Python, will be using something to accelerate things (PyPy, numpy, etc). Does, for example, numpy's sort() function suffer from the same flaw? Does PyPy use Timsort? Skip From no.email at nospam.invalid Tue Feb 24 18:53:55 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 24 Feb 2015 15:53:55 -0800 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> Message-ID: <8761aqamss.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > So why lock categorically like that? Java has a handy "synchronized" > keyword that wraps the whole method in "with self.lock". ... > Unfortunately, that deep locking analysis *is* required at every turn, > and mistakes *are* bound to happen. I wonder if synchronized was a mistake in Java. It confused me a lot when I tried to use it, but I never had to mess with it that much. I can't quite tell what your code is doing (why is it attempting a socket read with a lock held) and I'd be interested to see what an STM version would look like. From rosuav at gmail.com Tue Feb 24 19:07:52 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 11:07:52 +1100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: On Wed, Feb 25, 2015 at 10:50 AM, Skip Montanaro wrote: > Even if/when we get to the point where machines can hold an array of > 2**49 elements, I suspect people won't be using straight Python to > wrangle them. Looking just at CPython, what is the absolute minimum storage space for a massive list like that? My guess is that a 64-bit CPython might get as good as eight bytes per element; playing around with smaller figures (up to circa a million elements) shows it ranging from 8 to 9 bytes per element, bottoming out at just a smidge over 8, presumably at the moments when there's no slack space (there's a fixed overhead, which gets diminishingly significant). So an array of 2**49 elements would take 2**52 bytes (4 petabytes) of storage - addressable, to be sure, but an awful lot to be sorting. Would it be sufficient to stick a comment into the source saying "This may have problems with lists in excess of 2**49 elements", and leave it until that's actually likely to happen? ChrisA From breamoreboy at yahoo.co.uk Tue Feb 24 19:38:17 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 00:38:17 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 24/02/2015 22:36, sohcahtoa82 at gmail.com wrote: > On Tuesday, February 24, 2015 at 1:50:15 PM UTC-8, Mark Lawrence wrote: >> On 24/02/2015 21:34, Roy Smith wrote: >>> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >>> >> >> As you can clearly no longer rely on Python it looks like, after a long >> and loving relationship, I'm forced into moving on. PHP, Applescript, >> Javascript or back to C or even CORAL 66, what is your (plural) >> recommendation(s)? >> >> -- >> My fellow Pythonistas, ask not what our language can do for you, ask >> what you can do for our language. >> >> Mark Lawrence > > Personally, I'm going with BASIC being interpreted by a JavaScript script being fed from a PHP script running on an out-dated version of Apache. > > It includes an in-browser IDE that will be written as an ActiveX control and will require Internet Explorer 6. An Adobe Flash version is in the works to allow it to run on modern browsers. > Thanks for your help, much appreciated :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ckaynor at zindagigames.com Tue Feb 24 19:38:48 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Tue, 24 Feb 2015 16:38:48 -0800 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: On Tue, Feb 24, 2015 at 4:07 PM, Chris Angelico wrote: > On Wed, Feb 25, 2015 at 10:50 AM, Skip Montanaro > wrote: > > Even if/when we get to the point where machines can hold an array of > > 2**49 elements, I suspect people won't be using straight Python to > > wrangle them. > > Looking just at CPython, what is the absolute minimum storage space > for a massive list like that? My guess is that a 64-bit CPython might > get as good as eight bytes per element; playing around with smaller > figures (up to circa a million elements) shows it ranging from 8 to 9 > bytes per element, bottoming out at just a smidge over 8, presumably > at the moments when there's no slack space (there's a fixed overhead, > which gets diminishingly significant). So an array of 2**49 elements > would take 2**52 bytes (4 petabytes) of storage - addressable, to be > sure, but an awful lot to be sorting. > > Would it be sufficient to stick a comment into the source saying "This > may have problems with lists in excess of 2**49 elements", and leave > it until that's actually likely to happen? > I would agree that it will be quite a while as it stands before the bug appears, so documenting it MAY be appropriate, as it is likely to be far enough in the future that a number of unforeseen things may block the bug from being an issue (TimSort may be replaced by something better, CPython could die, etc). That said, from what I understand, their proposed fix would be reasonably easy to use, and have minimal cost, so it may be worthwhile to use just because the issue is likely to be in use for quite a while before somebody remembers the issue exists. Once somebody does hit it, it may take a while to realize that TimSort is at fault, as it will only occur on some data sets (my understanding is that the elements must be in certain configurations with at-least the minimum number), so it will likely appear as just a random crash occurring. If it were fully up to me, I would do one of the following, in order of preference: 1) Implement and test the fix provided on the blog. This requires the most work due to licensing, reviewing, and testing. 2) Add a noisy warning/error to the code when sorting lists large enough to potentially hit the error (probably 2**48 to leave some wiggle room). This is much easier to do, but merely pushes the can down the road to be dealt with later. It does still make it obvious what is going long when it finally breaks. 3) Add a comment explaining the issue (likely linking back to the blog). While by far the easiest "solution", again, this merely pushes the can down the road, and it may be very non-obvious what went wrong when it breaks, despite the comment. Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.stuart.85 at gmail.com Tue Feb 24 19:59:34 2015 From: ryan.stuart.85 at gmail.com (Ryan Stuart) Date: Wed, 25 Feb 2015 00:59:34 +0000 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> Message-ID: On Tue Feb 24 2015 at 3:32:47 PM Paul Rubin wrote: > Ryan Stuart writes: > Sure, the shared memory introduces the possibility of some bad errors, > I'm just saying that I've found that by staying with a certain > straightforward style, it doesn't seem difficult in practice to avoid > those errors. > And all I'm saying is that it doesn't matter how careful you are, all it takes is someone up the stack to not be as careful and you will be bitten. And even if they are careful, it's still very easy to be bitten because everything runs in shared memory. > I don't understand what you mean about malloc. > My point is malloc, something further up (down?) the stack, is making modifications to shared state when threads are involved. Modifying shared state makes it infinitely more difficult to reason about the correctness of your software. > That article was interesting in some ways but confused in others. One > way it was interesting is it said various non-thread approaches (such as > coroutines) had about the same problems as threads. Some ways it > was confused were: > We clearly got completely different things from the article. My interpretation was that it was making *the exact opposite* point to what you stated mainly because non-threading approaches don't share state. It states that quite clearly. For example "it is ? literally ? exponentially more difficult to reason about a routine that may be executed from an arbitrary number of threads concurrently". > 1) thinking Haskell threads were like processes with separate address > spaces. In fact they are in the same address space and programming > with them isn't all that different from Python threads, though the > synchronization primitives are a bit different. There is also an STM > library available that is ingenious though apparently somewhat slow. > I don't know Haskell, so I can't comment too much, except to say that by default Haskell looks to use lightweight threads where only 1 thread can be executing at a time [1] (sounds similar to the GIL to me). That doesn't seem to be shared state multithreading, which is what the article is referring to. But again, they will undoubtedly be someone sharing state higher up the stack. > 2) it has a weird story about the brass cockroach, that basically > signified that they didn't have a robust enough testing system to be > able to reproduce the bug. That is what they should have worked on. > The point was that it wasn't feasible to have a robust testing suite because, you guessed it, shared state and concurrent threads producing n*n complexity. > 3) It goes into various hazards of the balance transfer example not > mentioning that STM (available in Haskell and Clojure) completely > solves it. > This is probably correct. Is there any STM implementations out that that don't significantly compromise performance? Anyway, I got one thing out of this, which is that the multiprocessing > module looks pretty nice and I should try it even when I don't need > multicore parallelism, so thanks for that. > It's 1 real advantage is that it side-steps the GIL. So, if you need to utilise multiple cores for CPU bound tasks, then it might well be the only option. Cheers > In reality though, Python is still my most productive language for > throwaway, non-concurrent scripting, but for more complicated concurrent > programs, alternatives like Haskell, Erlang, and Go all have significant > attractions. > -- > https://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Tue Feb 24 20:19:42 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 25 Feb 2015 12:19:42 +1100 Subject: Newbie question about text encoding References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name><201502241945.t1OJjshO013092@fido.openend.se> Message-ID: <54ed232f$0$13004$c3e8da3$5496439d@news.astraweb.com> Laura Creighton wrote: > Dave Angel > are you another Native English speaker living in a world where ASCII > is enough? ASCII was never enough. Not even for Americans, who couldn't write things like "I bought a comic book for 10? yesterday", let alone interesting things from maths and science. I missed the whole 7-bit ASCII period, my first computer (Mac 128K) already had an extended character set beyond ASCII. But even that never covered the full range of characters I wanted to write, and then there was the horrible mess that you got whenever you copied text files from a Mac to a DOS or Windows PC or visa versa. Yes, even in 1984 we were transferring files and running into encoding issues. -- Steven From steve+comp.lang.python at pearwood.info Tue Feb 24 20:20:08 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 25 Feb 2015 12:20:08 +1100 Subject: Newbie question about text encoding References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name> Message-ID: <54ed2349$0$13004$c3e8da3$5496439d@news.astraweb.com> Laura Creighton wrote: > The idea that the whole world loves utf-8 is nonsense. I don't think anyone says the whole world loves UTF-8. I think people say that the whole world *ought to* love UTF-8, and that legacy encodings from the Windows "code-page" days ought to die. > Most of europe has been using latin1, latin2 etc. before > unicode was invented and will, as far as I know, continue to use it. And this is why people in Greece cannot transfer text files to people in France without the content changing (ISO-8859-7 vs ISO-8859-1). And why Russians cannot even swap text files with other Russians (a plethora of encodings). :-( -- Steven From tjreedy at udel.edu Tue Feb 24 20:29:15 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 24 Feb 2015 20:29:15 -0500 Subject: Running Idle on Windows 8 (.1) In-Reply-To: <0bbecc0a-9e03-4441-9fa8-c96bd3f690ec@googlegroups.com> References: <0bbecc0a-9e03-4441-9fa8-c96bd3f690ec@googlegroups.com> Message-ID: On 2/24/2015 3:13 PM, blakemalc66 at gmail.com wrote: > I too can not get idle to run on win 8.1 using python3.4.2 installed from the python-3.4.2.amd64.msi. What experience have others had with Idle and Windows 8? The OP for https://stackoverflow.com/questions/28633690/python-installation-troubleshooting wrote in a comment "I am invoking it from a win8 startmenu program, so I tried to do it via the normal windows 8 interface and it works from there," I do not know what 'normal windows 8 interface' means, but does it work for anyone else? > pythonw.exe appears in the task manager but no gui is displayed. pythonw.exe should appear *twice* in task manager. Please try 3.4.3rc1 and 3.5.0a1 https://www.python.org/downloads/windows/ If the Idle does not start correctly on Win 8, perhaps we could fix it before 3.4.3 final (which should be soon). -- Terry Jan Reedy From marfig at gmail.com Tue Feb 24 21:37:26 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Wed, 25 Feb 2015 03:37:26 +0100 Subject: py.test can't resolve imports Message-ID: I cannot seem to solve this problem from either the pytest documentation or from other places on the web which provide tutorials on pytest, such as http://pythontesting.net/start-here/ I have the following project setup: /project /project/project <-- script files (with __init__.py) /project/test <-- test files /project/docs <-- Sphinx I cannot seem to be able to have py.test resolve imports by running the following command from within the /project/project directory: $ py.test ../ Example scripts: /project/project/example.py def func(): pass /project/test/test_example.py from example import * def test_example(): assert 0 I get an error: $ py.test ../ from example import * E ImportError: No module named 'example' I can run the tests fine with $ python -m pytest ../ So how do I properly setup if I wish to use the less verbose py.test? From davea at davea.name Tue Feb 24 21:41:42 2015 From: davea at davea.name (Dave Angel) Date: Tue, 24 Feb 2015 21:41:42 -0500 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: <54ED3666.70000@davea.name> On 02/24/2015 07:07 PM, Chris Angelico wrote: > On Wed, Feb 25, 2015 at 10:50 AM, Skip Montanaro > wrote: >> Even if/when we get to the point where machines can hold an array of >> 2**49 elements, I suspect people won't be using straight Python to >> wrangle them. > > Looking just at CPython, what is the absolute minimum storage space > for a massive list like that? My guess is that a 64-bit CPython might > get as good as eight bytes per element; playing around with smaller > figures (up to circa a million elements) shows it ranging from 8 to 9 > bytes per element, bottoming out at just a smidge over 8, presumably > at the moments when there's no slack space (there's a fixed overhead, > which gets diminishingly significant). So an array of 2**49 elements > would take 2**52 bytes (4 petabytes) of storage - addressable, to be > sure, but an awful lot to be sorting. > > Would it be sufficient to stick a comment into the source saying "This > may have problems with lists in excess of 2**49 elements", and leave > it until that's actually likely to happen? > > ChrisA > One could do much better than that. Put in a test for the length of the list, and if it's too large for the current implementation, throw an exception. Much better than a comment. -- DaveA From tjreedy at udel.edu Tue Feb 24 21:52:25 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 24 Feb 2015 21:52:25 -0500 Subject: Bug in timsort!? In-Reply-To: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 2/24/2015 4:34 PM, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ Tim Peters is aware of this and opened http://bugs.python.org/issue23515 -- Terry Jan Reedy From bryceodell at icloud.com Tue Feb 24 21:55:09 2015 From: bryceodell at icloud.com (Audrey McFarlane) Date: Tue, 24 Feb 2015 19:55:09 -0700 Subject: python 2 to python 3 Message-ID: I am using Wing101 v.5 and it is using Python2, but I want to make it use Python3 instead because need Python3 for a uni lab. How do I change it? From marcos.al.azevedo at gmail.com Tue Feb 24 23:54:22 2015 From: marcos.al.azevedo at gmail.com (Marcos Almeida Azevedo) Date: Wed, 25 Feb 2015 12:54:22 +0800 Subject: Newbie question about text encoding In-Reply-To: <54ed232f$0$13004$c3e8da3$5496439d@news.astraweb.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <54ed232f$0$13004$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 25, 2015 at 9:19 AM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > Laura Creighton wrote: > > > Dave Angel > > are you another Native English speaker living in a world where ASCII > > is enough? > > ASCII was never enough. Not even for Americans, who couldn't write things > like "I bought a comic book for 10? yesterday", let alone interesting > things from maths and science. > > ASCII was a necessity back then because RAM and storage are too small. > I missed the whole 7-bit ASCII period, my first computer (Mac 128K) already > had an extended character set beyond ASCII. But even that never covered the > I miss the days when I was coding with my XT computer (640kb RAM) too. Things were so simple back then. > full range of characters I wanted to write, and then there was the horrible > mess that you got whenever you copied text files from a Mac to a DOS or > Windows PC or visa versa. Yes, even in 1984 we were transferring files and > running into encoding issues. > > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java -------------- next part -------------- An HTML attachment was scrubbed... URL: From marko at pacujo.net Wed Feb 25 00:25:10 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 25 Feb 2015 07:25:10 +0200 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> Message-ID: <87zj82bm15.fsf@elektro.pacujo.net> Paul Rubin : > Marko Rauhamaa writes: >> So why lock categorically like that? Java has a handy "synchronized" >> keyword that wraps the whole method in "with self.lock". ... >> Unfortunately, that deep locking analysis *is* required at every turn, >> and mistakes *are* bound to happen. > > I wonder if synchronized was a mistake in Java. It confused me a lot > when I tried to use it, but I never had to mess with it that much. I > can't quite tell what your code is doing (why is it attempting a > socket read with a lock held) and I'd be interested to see what an STM > version would look like. Synchronized methods are actually quite a powerful idea, but a bit underappreciated by many Java developers. The main point of my example is this: * Effective thread-safe programming would require some guidelines, a routine, a methodology so the code would be free of locking anomalies as a matter of course. * The industry hasn't formed such guidelines that are generally followed. That makes it difficult to glue together systems from components that come from different sources. * Even the best of guidelines have significant, surprising corner cases that require special treatment and are easy to get wrong. * As a corollary, you cannot implement thread-safety locally in a class. Instead, your locking decisions must be made application-wide. That violates encapsulation, which is a cornerstone of object-oriented programming (and large-system manageability). Marko From marcos.al.azevedo at gmail.com Wed Feb 25 00:34:33 2015 From: marcos.al.azevedo at gmail.com (Marcos Almeida Azevedo) Date: Wed, 25 Feb 2015 13:34:33 +0800 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: <87zj82bm15.fsf@elektro.pacujo.net> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 25, 2015 at 1:25 PM, Marko Rauhamaa wrote: > Paul Rubin : > > > Marko Rauhamaa writes: > >> So why lock categorically like that? Java has a handy "synchronized" > >> keyword that wraps the whole method in "with self.lock". ... > >> Unfortunately, that deep locking analysis *is* required at every turn, > >> and mistakes *are* bound to happen. > > > > I wonder if synchronized was a mistake in Java. It confused me a lot > > when I tried to use it, but I never had to mess with it that much. I > > can't quite tell what your code is doing (why is it attempting a > > socket read with a lock held) and I'd be interested to see what an STM > > version would look like. > > Synchronized methods are actually quite a powerful idea, but a bit > underappreciated by many Java developers. > > Synchronized methods in Java really makes programming life simpler. But I think it is standard practice to avoid this if there is a lighter alternative as synchronized methods are slow. Worse case I used double checked locking. > > > Marko > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java -------------- next part -------------- An HTML attachment was scrubbed... URL: From marko at pacujo.net Wed Feb 25 00:46:47 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 25 Feb 2015 07:46:47 +0200 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> Message-ID: <87pp8ybl14.fsf@elektro.pacujo.net> Marcos Almeida Azevedo : > Synchronized methods in Java really makes programming life simpler. > But I think it is standard practice to avoid this if there is a > lighter alternative as synchronized methods are slow. Worse case I > used double checked locking. I have yet to see code whose performance suffers from too much locking. However, I have seen plenty of code that suffers from anomalies caused by incorrect locking. Marko From steve+comp.lang.python at pearwood.info Wed Feb 25 00:47:55 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 25 Feb 2015 16:47:55 +1100 Subject: Pyston 0.3 self-hosting Message-ID: <54ed620d$0$11100$c3e8da3@news.astraweb.com> Pyston 0.3, the latest version of a new high-performance Python implementation, has reached self-hosting sufficiency: http://blog.pyston.org/2015/02/24/pyston-0-3-self-hosting-sufficiency/ -- Steve From rosuav at gmail.com Wed Feb 25 00:54:08 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 16:54:08 +1100 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: <87pp8ybl14.fsf@elektro.pacujo.net> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 25, 2015 at 4:46 PM, Marko Rauhamaa wrote: > Marcos Almeida Azevedo : > >> Synchronized methods in Java really makes programming life simpler. >> But I think it is standard practice to avoid this if there is a >> lighter alternative as synchronized methods are slow. Worse case I >> used double checked locking. > > I have yet to see code whose performance suffers from too much locking. > However, I have seen plenty of code that suffers from anomalies caused > by incorrect locking. Uhh, I have seen *heaps* of code whose performance suffers from too much locking. At the coarsest and least intelligent level, a database program that couldn't handle concurrency at all, so I wrote an application-level semaphore that stopped two people from running it at once. You want to use that program? Ask the other guy to close it. THAT is a performance problem. And there are plenty of narrower cases, where it ends up being a transactions-per-second throughput limiter. ChrisA From marcos.al.azevedo at gmail.com Wed Feb 25 00:58:02 2015 From: marcos.al.azevedo at gmail.com (Marcos Almeida Azevedo) Date: Wed, 25 Feb 2015 13:58:02 +0800 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: <87pp8ybl14.fsf@elektro.pacujo.net> References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 25, 2015 at 1:46 PM, Marko Rauhamaa wrote: > Marcos Almeida Azevedo : > > > Synchronized methods in Java really makes programming life simpler. > > But I think it is standard practice to avoid this if there is a > > lighter alternative as synchronized methods are slow. Worse case I > > used double checked locking. > > I have yet to see code whose performance suffers from too much locking. > However, I have seen plenty of code that suffers from anomalies caused > by incorrect locking. > > > Of course code with locking is slower than one without. The locking mechanism itself is the overhead. So use it only when necessary. There is a reason why double checked locking was invented by clever programmers. > Marko > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java -------------- next part -------------- An HTML attachment was scrubbed... URL: From ian.g.kelly at gmail.com Wed Feb 25 01:02:12 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 24 Feb 2015 23:02:12 -0700 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Tue, Feb 24, 2015 at 10:54 PM, Chris Angelico wrote: > On Wed, Feb 25, 2015 at 4:46 PM, Marko Rauhamaa wrote: >> Marcos Almeida Azevedo : >> >>> Synchronized methods in Java really makes programming life simpler. >>> But I think it is standard practice to avoid this if there is a >>> lighter alternative as synchronized methods are slow. Worse case I >>> used double checked locking. >> >> I have yet to see code whose performance suffers from too much locking. >> However, I have seen plenty of code that suffers from anomalies caused >> by incorrect locking. > > Uhh, I have seen *heaps* of code whose performance suffers from too > much locking. At the coarsest and least intelligent level, a database > program that couldn't handle concurrency at all, so I wrote an > application-level semaphore that stopped two people from running it at > once. You want to use that program? Ask the other guy to close it. > THAT is a performance problem. And there are plenty of narrower cases, > where it ends up being a transactions-per-second throughput limiter. Is the name of that database program "Microsoft Access" perchance? From rosuav at gmail.com Wed Feb 25 01:07:35 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 17:07:35 +1100 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 25, 2015 at 5:02 PM, Ian Kelly wrote: >> Uhh, I have seen *heaps* of code whose performance suffers from too >> much locking. At the coarsest and least intelligent level, a database >> program that couldn't handle concurrency at all, so I wrote an >> application-level semaphore that stopped two people from running it at >> once. You want to use that program? Ask the other guy to close it. >> THAT is a performance problem. And there are plenty of narrower cases, >> where it ends up being a transactions-per-second throughput limiter. > > Is the name of that database program "Microsoft Access" perchance? No, though it wouldn't surprise me if it had the same issue. No, the program was a DBase-backed one of my own development; it was the DBase engine itself that couldn't handle concurrency, so I added some startup code that checked to see if anyone else had the file open, and popped up a "retry or cancel" prompt until the semaphore cleared. Later on, I was able to shift the entire content into DB2 (once we no longer needed compatibility with an even older DBase-backed program), and voila, we had concurrency. I still needed to make use of record-level locking (if you open a record for editing, it holds a lock for as long as you have the window open; chances are, anyone else who wants the same record is actually doing the same job, so erroring out is the best option), but no more database-level locks. ChrisA From spande04 at gmail.com Wed Feb 25 02:42:42 2015 From: spande04 at gmail.com (Swapnil Pande) Date: Tue, 24 Feb 2015 23:42:42 -0800 (PST) Subject: progress bar Message-ID: <981a6a04-b505-4d6c-8c58-935827195a90@googlegroups.com> i want to call another tkinter window after completing the progress bar an n e one help me From mail at timgolden.me.uk Wed Feb 25 03:26:18 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 25 Feb 2015 08:26:18 +0000 Subject: Python Silent Install In-Reply-To: <1424819114.89841.YahooMailMobile@web163806.mail.gq1.yahoo.com> References: <1424819114.89841.YahooMailMobile@web163806.mail.gq1.yahoo.com> Message-ID: <54ED872A.7050801@timgolden.me.uk> [... re installing with ensurepip disabled ...] On 24/02/2015 23:05, Albert-Jan Roskam wrote: > Personally I find that Python is incomplete without pip and setuptools. Of course; that's why the ensurepip was added to the installers. But there are other ways of installing pip after the event. Including simply: py -3 -m ensurepip TJG From blakemalc66 at gmail.com Wed Feb 25 03:34:33 2015 From: blakemalc66 at gmail.com (blakemalc66 at gmail.com) Date: Wed, 25 Feb 2015 00:34:33 -0800 (PST) Subject: Compatibility Issues - Windows 8 In-Reply-To: References: Message-ID: On Tuesday, February 24, 2015 at 2:57:08 PM UTC+11, Iris J Rosario wrote: > Hello, > > > > I've been having problems with running python IDLE on my Windows 8 tower. Can you please let me know where I can rea up on how to correct it? Or which version will work best on my computer? > > > > Best Regards, > > Iris J Rosario > > ijro... at students.pccc.edu > > > > "One can complain because rosebushes have thorns, or rejoice because thorn bushes have roses." ~ Abraham Lincoln > > Terry I am downloading 3.4.3rc1 now and will install in the morning. My task manager showed 2 pythonw.exe entries. I tried executing from the win8.1 metro icon, and also from within the idlelib: idle.bat, idle.py and idle.pyw. All with the same result - no gui Malcolm From ben+python at benfinney.id.au Wed Feb 25 03:36:55 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 25 Feb 2015 19:36:55 +1100 Subject: python 2 to python 3 References: Message-ID: <851tle75g8.fsf@benfinney.id.au> Audrey McFarlane writes: > I am using Wing101 v.5 and it is using Python2, but I want to make it > use Python3 instead because need Python3 for a uni lab. How do I > change it? You'll need to use a different version of that program. Does Wing101 have a version that runs with Python 3? If so, get that version and install it. If not, agitate with the vendor to create and release such a version. -- \ ?What is needed is not the will to believe but the will to find | `\ out, which is the exact opposite.? ?Bertrand Russell, _Free | _o__) Thought and Official Propaganda_, 1928 | Ben Finney From nad at acm.org Wed Feb 25 03:39:54 2015 From: nad at acm.org (Ned Deily) Date: Wed, 25 Feb 2015 00:39:54 -0800 Subject: Python shell: Arrow keys not working in PuTTY References: <41302A7145AC054FA7A96CFD03835A0A0B9451B8@EX10MBX02.EU.NEC.COM> <54ebdcfa$0$11100$c3e8da3@news.astraweb.com> <54ec1360$0$12978$c3e8da3$5496439d@news.astraweb.com> Message-ID: In article <54ec1360$0$12978$c3e8da3$5496439d at news.astraweb.com>, Steven D'Aprano wrote: > Ned Deily wrote: > > With no --prefix= on ./configure, the default install location is to > > /usr/local, so "make install" would install a link at > > /usr/local/bin/python (or python3) and it would only overwrite your > > system Python if the system Python happened to be installed in > > /usr/local/bin/. > Well, I'm not going to say you are wrong, but I can say that on Centos 5 > systems (and presumably that includes Fedora and Red Hat of the equivalent > vintage), if you just run `make install` it overwrites the /usr/bin/python > hard link to /usr/bin/python2.4. I know because I've done it :-( If you're using any current 2.7.x or 3.4.x source tarball (or dev repo) from python.org, it ain't gonna happen. (I'm not going to go back and check all the previous releases but it's been that way for a long time, AFAIK.) If you're using source modified by a distribution, like Centos, RH, or Fedora, all bets are off, of course. -- Ned Deily, nad at acm.org From nad at acm.org Wed Feb 25 04:04:36 2015 From: nad at acm.org (Ned Deily) Date: Wed, 25 Feb 2015 01:04:36 -0800 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: In article , Chris Angelico wrote: > The post links to: > > http://svn.python.org/projects/python/trunk/Objects/listobject.c > > Is that still valid and current? It says svn, but does it pop through > to the Mercurial repo where the _real_ CPython code is stored? In general, no. The python svn repo linked to above is frozen in time: for most branches, as of the conversion to hg in 2011. There was at least one discussion thread soon thereafter about what to do about it: http://comments.gmane.org/gmane.comp.python.devel/125252 It is still true that at least one repo in the svn.python.org is actively maintained (e.g. "external"). I *think* all the others are now obsolete. http://svn.python.org/view/ -- Ned Deily, nad at acm.org From paddy3118 at gmail.com Wed Feb 25 04:10:32 2015 From: paddy3118 at gmail.com (Paddy) Date: Wed, 25 Feb 2015 01:10:32 -0800 (PST) Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: On Wednesday, 25 February 2015 00:08:32 UTC, Chris Angelico wrote: > On Wed, Feb 25, 2015 at 10:50 AM, Skip Montanaro > wrote: > > Even if/when we get to the point where machines can hold an array of > > 2**49 elements, I suspect people won't be using straight Python to > > wrangle them. > <> > > Would it be sufficient to stick a comment into the source saying "This > may have problems with lists in excess of 2**49 elements", and leave > it until that's actually likely to happen? > > ChrisA If we are given a proven fix with little downside then if we are to touch the source then we should MAKE THE FIX. To do otherwise would send the wrong signal to those that depend on the perceived integrity of the C-Python developers. From lk.palao at gmail.com Wed Feb 25 04:16:55 2015 From: lk.palao at gmail.com (Leo Kris Palao) Date: Wed, 25 Feb 2015 17:16:55 +0800 Subject: Configuring problems with GDAL in enthought Python Canopy Message-ID: Hi ALL, Just wanted to ask if somebody could guide me in installing GDAL in my Python installed using Canopy. Could you give me some steps how to successfully install this package? I got it running using my previous Python Installation, but I removed it and used Canopy Python now. btw: my python installation is located in: C:\Users\lpalao\AppData\Local\Enthought\Canopy\User\Scripts\python.exe Thanks in advance for the help. -Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Wed Feb 25 05:11:55 2015 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Feb 2015 21:11:55 +1100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ED0B7E.7030801@mrabarnett.plus.com> Message-ID: On Wed, Feb 25, 2015 at 8:10 PM, Paddy wrote: > If we are given a proven fix *with little downside* then if we are to touch the source then we should MAKE THE FIX. (emphasis mine) That's really the question, though. I'm in no position to evaluate the patch and ascertain whether the fix does indeed have little or no downside. But as the tracker issue proves, a greater mind than mine has decided that it's to be applied, which means that, yes, it's worth doing. ChrisA From steve+comp.lang.python at pearwood.info Wed Feb 25 05:31:04 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 25 Feb 2015 21:31:04 +1100 Subject: python 2 to python 3 References: Message-ID: <54eda468$0$12993$c3e8da3$5496439d@news.astraweb.com> Audrey McFarlane wrote: > I am using Wing101 v.5 and it is using Python2, but I want to make it use > Python3 instead because need Python3 for a uni lab. How do I change it? I'm afraid I don't use Wing so I can't give a good answer, but I googled and found this: http://stackoverflow.com/questions/25318792/how-do-i-configure-wing101-to-run-python-3-3 It's not very helpful, but it might point you in the right direction. If you do work out the answer, please tell us, so that the next person who asks can get an answer too. -- Steven From steve+comp.lang.python at pearwood.info Wed Feb 25 05:33:13 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 25 Feb 2015 21:33:13 +1100 Subject: progress bar References: <981a6a04-b505-4d6c-8c58-935827195a90@googlegroups.com> Message-ID: <54eda4ea$0$12988$c3e8da3$5496439d@news.astraweb.com> Swapnil Pande wrote: > i want to call another tkinter window after completing the progress bar > an n e one help me What does "call another tkinter window" mean? I understand "call another function", but how do you call a window? Perhaps it will help if you show us some code. -- Steven From rustompmody at gmail.com Wed Feb 25 06:03:45 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 25 Feb 2015 03:03:45 -0800 (PST) Subject: strip bug? In-Reply-To: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: <26f2f167-eead-4966-b567-f18eb4975bd2@googlegroups.com> On Wednesday, February 25, 2015 at 4:35:37 AM UTC+5:30, bay... at gmail.com wrote: > >>> 'http://xthunder'.strip('http://') > 'xthunder' > >>> 'http://thunder'.strip('http://') > 'under' > >>> > > I could understand backslash but forward slash? Others have answered specifically. However you probably want to look at urlparse https://docs.python.org/2/library/urlparse.html From robert.kern at gmail.com Wed Feb 25 06:34:29 2015 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Feb 2015 11:34:29 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 2015-02-24 22:45, Grant Edwards wrote: > On 2015-02-24, Roy Smith wrote: > >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > > I don't get it. > > 3.2 Corrected Python merge_collapse function > > merge_collapse(MergeState *ms) > { > struct s_slice *p = ms->pending; > > assert(ms); > while (ms->n > 1) { > Py_ssize_t n = ms->n - 2; > if ( n > 0 && p[n-1].len <= p[n].len + p[n+1].len > || (n-1 > 0 && p[n-2].len <= p[n].len + p[n-1].len)) { > if (p[n-1].len < p[n+1].len) > --n; > if (merge_at(ms, n) < 0) > return -1; > } > else if (p[n].len <= p[n+1].len) { > if (merge_at(ms, n) < 0) > return -1; > } > else > break; > } > return 0; > } > > Or does "Python function" mean something else in this context? "Corrected merge_collapse function [from the Python implementation of TimSort]" as opposed to the Java implementation which was also discussed. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From cfkaran2 at gmail.com Wed Feb 25 07:46:19 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Wed, 25 Feb 2015 07:46:19 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <575FE33B-D1AD-4391-B816-7440D5849C41@gmail.com> On Feb 24, 2015, at 8:23 AM, Fabio Zadrozny wrote: > Hi Cem, > > I didn't read the whole long thread, but I thought I'd point you to what I'm using in PyVmMonitor (http://www.pyvmmonitor.com/) -- which may already cover your use-case. > > Take a look at the callback.py at https://github.com/fabioz/pyvmmonitor-core/blob/master/pyvmmonitor_core/callback.py > > And its related test (where you can see how to use it): https://github.com/fabioz/pyvmmonitor-core/blob/master/_pyvmmonitor_core_tests/test_callback.py (note that it falls back to a strong reference on simple functions -- i.e.: usually top-level methods or methods created inside a scope -- but otherwise uses weak references). That looks like a better version of what I was thinking about originally. However, various people on the list have convinced me to stick with strong references everywhere. I'm working out a possible API right now, once I have some code that I can use to illustrate what I'm thinking to everyone, I'll post it to the list. Thank you for showing me your code though, it is clever! Thanks, Cem Karan From cfkaran2 at gmail.com Wed Feb 25 07:50:02 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Wed, 25 Feb 2015 07:50:02 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <8621FCE6-30A9-4DBF-B398-4DD5CF42FFE8@gmail.com> On Feb 24, 2015, at 4:19 PM, Gregory Ewing wrote: > random832 at fastmail.us wrote: >> On Tue, Feb 24, 2015, at 00:20, Gregory Ewing wrote: >>> This is why I suggested registering a listener object >>> plus a method name instead of a callback. It avoids that >>> reference cycle, because there is no long-lived callback >>> object keeping a reference to the listener. >> How does that help? Everywhere you would have had a reference to the >> "callback object", you now have a reference to the listener object. > > The point is that the library can keep a weak reference > to the listener object, whereas it can't reliably keep > a weak reference to a bound method. I think I see what you're talking about now. Does WeakMethod (https://docs.python.org/3/library/weakref.html#weakref.WeakMethod) solve this problem? Note that I can force my users to use the latest stable version of python at all times, so WeakMethod IS available to me. Thanks, Cem Karan From larry at hastings.org Wed Feb 25 08:07:08 2015 From: larry at hastings.org (Larry Hastings) Date: Wed, 25 Feb 2015 05:07:08 -0800 Subject: [RELEASED] Python 3.4.3 is now available Message-ID: <54EDC8FC.5040003@hastings.org> On behalf of the Python development community and the Python 3.4 release team, I'm pleased to announce the availability of Python 3.4.3. Python 3.4.3 has many bugfixes and other small improvements over 3.4.2. You can find it here: https://www.python.org/downloads/release/python-343/ The release slipped by two or three days, depending on what time zone you're in. This is my fault--I apologize for the inconvenience. Cheers, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wrw at mac.com Wed Feb 25 08:51:21 2015 From: wrw at mac.com (William Ray Wing) Date: Wed, 25 Feb 2015 08:51:21 -0500 Subject: python 2 to python 3 In-Reply-To: References: Message-ID: <8FF09802-F13B-4000-ACA8-0CFCD8D45E38@mac.com> > On Feb 24, 2015, at 9:55 PM, Audrey McFarlane wrote: > > I am using Wing101 v.5 and it is using Python2, but I want to make it use Python3 instead because need Python3 for a uni lab. How do I change it? > -- > https://mail.python.org/mailman/listinfo/python-list Assuming you have Python3 installed on your system. The most straightforward way to make Wing use it is to go to the Project menu, select Project Properties, and under the Environment tab, Python Executable, click the custom button, then enter the path to the Python3 executable in the path window. Have fun, Bill From sturla.molden at gmail.com Wed Feb 25 08:58:31 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 25 Feb 2015 14:58:31 +0100 Subject: Bug in timsort!? In-Reply-To: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 24/02/15 22:34, Roy Smith wrote: > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ > This is awful. It is broken for arrays longer than 2**49 elements. With 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't see how we can fix this in time. Oh yes, and they mention that TimSort is used on billions of devices due to Android mobile phones. This is clearly very relevant for mobile phones. Next thing you know your litte Samsung Galaxy with more than 4096 terabytes breaks down from a stack overflow in TimSort. Sturla From jonas at wielicki.name Wed Feb 25 09:07:57 2015 From: jonas at wielicki.name (Jonas Wielicki) Date: Wed, 25 Feb 2015 15:07:57 +0100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: <54EDD73D.8080905@wielicki.name> On 25.02.2015 14:58, Sturla Molden wrote: > On 24/02/15 22:34, Roy Smith wrote: >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >> >> > [?] > > Oh yes, and they mention that TimSort is used on billions of devices due > to Android mobile phones. This is clearly very relevant for mobile > phones. Next thing you know your litte Samsung Galaxy with more than > 4096 terabytes breaks down from a stack overflow in TimSort. The Java version of the bug is reproducible with just 67108864 elements, if I read the article correctly. jwi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From rosuav at gmail.com Wed Feb 25 09:33:54 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 01:33:54 +1100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Thu, Feb 26, 2015 at 12:58 AM, Sturla Molden wrote: > This is awful. It is broken for arrays longer than 2**49 elements. With 8 > bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't see how > we can fix this in time. It's even worse than that. Unless you have a list of 2**49 references to the same few objects, you're going to need to have some actual content for each one. The absolute best you could do is to sort integers, which would take 32 bytes each [1]; if you're sorting strings, they'll be 90 bytes each, so the integers are our best bet. So add another *five* powers of two to the RAM requirements. Also, who has that many elements to sort, and wouldn't do better to stick 'em into an indexed database of some description? I mean, good to know that this'll be fixed, but it really isn't likely to come up any time soon. ChrisA [1] Calculated using Python 3.5 amd64 Linux, your figures may vary. Small integers (less than 2**30) take only 28 bytes, but that's dwarfed by the rest of them taking 32. From wxjmfauth at gmail.com Wed Feb 25 09:34:40 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 25 Feb 2015 06:34:40 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: <54ed2349$0$13004$c3e8da3$5496439d@news.astraweb.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name> <54ed2349$0$13004$c3e8da3$5496439d@news.astraweb.com> Message-ID: <52672557-6aa2-4cf4-a61c-7139cdca9cb0@googlegroups.com> ======================== U+0001F601 U+0001F602 ... From fomcl at yahoo.com Wed Feb 25 09:43:28 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 25 Feb 2015 14:43:28 +0000 (UTC) Subject: mx as dependency tox throws GCC error Message-ID: <1569855376.2617820.1424875408718.JavaMail.yahoo@mail.yahoo.com> Hi, If I pip install the mx package with "pip install egenix-mx-base", it works. If I put that same pip install command under 'install_command' in my tox.ini it also works (see below) However, if I specify the dependency under 'deps', I get an error. Any idea why? I read that 'install_command' is still experimental and subject to change, so I would prefer to specify the package under 'deps'. The error occurred when I called tox using 'tox -e py27', so it is not pypy-related. [tox] envlist = pypy,py27,py33,py34,docs skipsdist = true [testenv] deps = setuptools nose gocept.testing numpy cython # egenix-mx-base # --> mx tox gcc: error: mx/DateTime/mxDateTime/mxDateTime.c: No such file or directory install_command = pip install egenix-mx-base {packages} Thanks! Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From mal at europython.eu Wed Feb 25 09:50:08 2015 From: mal at europython.eu (M.-A. Lemburg) Date: Wed, 25 Feb 2015 15:50:08 +0100 Subject: EuroPython 2015: Our first keynote speaker: Guido van Rossum Message-ID: <54EDE120.4040208@europython.eu> We are pleased to announce our first keynote speaker for EuroPython 2015: Python?s creator: Guido van Rossum Guido will give a keynote and a more technical talk about the new type hinting proposal for Python 3.5 that?s currently being discussed as PEP 483 (The Theory of Type Hints) and PEP 484 (Type Hints). Enjoy, - EuroPython Society (EPS) http://www.europython-society.org/ PS: We have more exciting news coming in the next few days. Please watch our blog for updates: http://blog.europython.eu/ From alister.nospam.ware at ntlworld.com Wed Feb 25 10:03:49 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Wed, 25 Feb 2015 15:03:49 +0000 (UTC) Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, 25 Feb 2015 14:58:31 +0100, Sturla Molden wrote: > On 24/02/15 22:34, Roy Smith wrote: >> http://envisage-project.eu/proving-android-java-and-python-sorting- algorithm-is-broken-and-how-to-fix-it/ >> >> > This is awful. It is broken for arrays longer than 2**49 elements. With > 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't > see how we can fix this in time. > > Oh yes, and they mention that TimSort is used on billions of devices due > to Android mobile phones. This is clearly very relevant for mobile > phones. Next thing you know your litte Samsung Galaxy with more than > 4096 terabytes breaks down from a stack overflow in TimSort. > > > Sturla the oh well it wont affect current devices attitude is a very poor attitude to big fixes. if a bug is reported it should be resolved a quickly as practical sot hat it never does become a real world issue much better to bolt the door BEFORE the horse bolts (even if the horse has not actually been borne yet) -- 1: No code table for op: ++post From sturla.molden at gmail.com Wed Feb 25 10:05:19 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 25 Feb 2015 16:05:19 +0100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/15 15:33, Chris Angelico wrote: > It's even worse than that. Unless you have a list of 2**49 references > to the same few objects, you're going to need to have some actual > content for each one. The absolute best you could do is to sort > integers, which would take 32 bytes each [1]; if you're sorting > strings, they'll be 90 bytes each, so the integers are our best bet. > So add another *five* powers of two to the RAM requirements. In that case you also need to add the PyObject_HEAD overhead for each object. ;-) Sturla From rosuav at gmail.com Wed Feb 25 10:11:20 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 02:11:20 +1100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Thu, Feb 26, 2015 at 2:05 AM, Sturla Molden wrote: > On 25/02/15 15:33, Chris Angelico wrote: > >> It's even worse than that. Unless you have a list of 2**49 references >> to the same few objects, you're going to need to have some actual >> content for each one. The absolute best you could do is to sort >> integers, which would take 32 bytes each [1]; if you're sorting >> strings, they'll be 90 bytes each, so the integers are our best bet. >> So add another *five* powers of two to the RAM requirements. > > > In that case you also need to add the PyObject_HEAD overhead for each > object. ;-) Not sure about that, because the figure of 32 bytes came from sys.getsizeof(). So assuming there's no significant malloc overhead (there shouldn't be any alignment padding, for instance), 32 bytes ought to be it - pack 'em all in. ChrisA From zachary.ware+pylist at gmail.com Wed Feb 25 10:23:15 2015 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Wed, 25 Feb 2015 09:23:15 -0600 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, Feb 25, 2015 at 9:03 AM, alister wrote: > the oh well it wont affect current devices attitude is a very poor > attitude to big fixes. > if a bug is reported it should be resolved a quickly as practical sot hat > it never does become a real world issue > > much better to bolt the door BEFORE the horse bolts (even if the horse > has not actually been borne yet) Just to be clear, this has already been fixed, and the fix will be released in Python 2.7.10, 3.4.4, and 3.5.0. -- Zach From invalid at invalid.invalid Wed Feb 25 10:49:12 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 25 Feb 2015 15:49:12 +0000 (UTC) Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 2015-02-25, Robert Kern wrote: > On 2015-02-24 22:45, Grant Edwards wrote: >> On 2015-02-24, Roy Smith wrote: >> >>> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >> >> I don't get it. >> >> 3.2 Corrected Python merge_collapse function [C code elided] >> Or does "Python function" mean something else in this context? > > "Corrected merge_collapse function [from the Python implementation of TimSort]" > as opposed to the Java implementation which was also discussed. Yes, I get it now. But, when I read "Python function" or "Python implementation of " that _to_me_ refers to someting written in _Python_. If you're talking about _CPython_ code, then you say "CPython function" or "CPython implementation of ". -- Grant Edwards grant.b.edwards Yow! I hope the at ``Eurythmics'' practice gmail.com birth control ... From ian.g.kelly at gmail.com Wed Feb 25 10:49:31 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Feb 2015 08:49:31 -0700 Subject: Running Idle on Windows 8 (.1) In-Reply-To: References: <0bbecc0a-9e03-4441-9fa8-c96bd3f690ec@googlegroups.com> Message-ID: On Tue, Feb 24, 2015 at 6:29 PM, Terry Reedy wrote: > On 2/24/2015 3:13 PM, blakemalc66 at gmail.com wrote: > >> I too can not get idle to run on win 8.1 using python3.4.2 installed from >> the python-3.4.2.amd64.msi. > > > What experience have others had with Idle and Windows 8? > > The OP for > https://stackoverflow.com/questions/28633690/python-installation-troubleshooting > > wrote in a comment "I am invoking it from a win8 startmenu program, so I > tried to do it via the normal windows 8 interface and it works from there," > > I do not know what 'normal windows 8 interface' means, but does it work for > anyone else? > >> pythonw.exe appears in the task manager but no gui is displayed. > > > pythonw.exe should appear *twice* in task manager. > Please try 3.4.3rc1 and 3.5.0a1 > https://www.python.org/downloads/windows/ > > If the Idle does not start correctly on Win 8, perhaps we could fix it > before 3.4.3 final (which should be soon). I just tried both of those (32-bit only) in a Windows 8.1 VM and had no problems starting Idle. From __peter__ at web.de Wed Feb 25 11:04:10 2015 From: __peter__ at web.de (Peter Otten) Date: Wed, 25 Feb 2015 17:04:10 +0100 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: Sturla Molden wrote: > On 24/02/15 22:34, Roy Smith wrote: >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >> > > This is awful. It is broken for arrays longer than 2**49 elements. With > 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't > see how we can fix this in time. > > Oh yes, and they mention that TimSort is used on billions of devices due > to Android mobile phones. This is clearly very relevant for mobile > phones. Next thing you know your litte Samsung Galaxy with more than > 4096 terabytes breaks down from a stack overflow in TimSort. Yeah, I'm looking forward to see comparable bugs being closed as "cannot reproduce" by some of the jokers. I suppose that you all had a note scribbled on the margin of your copy of Arithmetica that this cannot happen with arrays of lengths seen in practice until 2038 or so. These guys found a bug that is subtler than what most of us have dealt with in a widely used piece of code originally developed by one of the smarter members of the python "community". I bow my head to them and say thank you. From shalini.ravishankar at gmail.com Wed Feb 25 11:20:19 2015 From: shalini.ravishankar at gmail.com (Shalini Ravishankar) Date: Wed, 25 Feb 2015 08:20:19 -0800 (PST) Subject: Issue with VTK quadratic Python Program Message-ID: Hello Everyone, I am new to VTK. I m trying to create a quadratic function with interactive slider. But the output is not as expected. Can someone tell em what I am doing wrong here ? # First, we need to import vtk package in order to access VTK classes/functions. import vtk # create a data source...an implicit function. # F(x,y,z) = a0*x^2 + a1*y^2 + a2*z^2 + a3*x*y + a4*y*z + a5*x*z + a6*x + a7*y + a8*z + a9 # F(x,y,z) = 1*x^2 + 0.5*y^2 + 0.2*z^2 + 0*x*y + 0*y*z + 0.1*x*z + 0.2*x + 0*y + 0*z + 0 quadric = vtk.vtkQuadric() quadric.SetCoefficients(1, 0.5, 0.2, 0, 0, 0.1, 0.2, 0, 0, 0) def vtkSliderCallback2(obj, event): sliderRepres = obj.GetRepresentation() pos = sliderRepres.GetValue() print "Position ",pos isosurface.SetValue(0, pos) # create a filter...a sampling function, which samples an implicit function over the x-y-z range # although this object is not called "filter" it takes an input and do something to/with it # and produce an output. sample = vtk.vtkSampleFunction() sample.SetSampleDimensions(100, 100, 100) sample.SetImplicitFunction(quadric) outline = vtk.vtkOutlineFilter() outline.SetInput( sample.GetOutput() ) outlineMapper = vtk.vtkPolyDataMapper() outlineMapper.SetInput( outline.GetOutput() ) outlineActor = vtk.vtkActor() outlineActor.SetMapper( outlineMapper ) outlineActor.GetProperty().SetColor(0.0,0.0,1.0) # create another filter...computing a contour of an input data. isosurface = vtk.vtkContourFilter() isosurface.SetInputConnection(sample.GetOutputPort()) isosurface.GenerateValues(15, 0.0, 4.2) # create a mapper, which mapps data to visualizable data structure. contMapper = vtk.vtkPolyDataMapper() contMapper.SetInputConnection(isosurface.GetOutputPort()) contMapper.SetScalarRange(0.0, 1.2) # create an actor, which can be displayed. contActor = vtk.vtkActor() contActor.SetMapper(contMapper) # create a window with a renderer. ren = vtk.vtkRenderer() renWin = vtk.vtkRenderWindow() renWin.AddRenderer(ren) iren = vtk.vtkRenderWindowInteractor() iren.SetRenderWindow(renWin) ren.SetBackground(0.95, 0.95, 1.0) ren.AddActor(contActor) renWin.SetSize( 500, 500 ) SliderRepres = vtk.vtkSliderRepresentation2D() min = 0 #ImageViewer.GetSliceMin() max = 256 #ImageViewer.GetSliceMax() SliderRepres.SetMinimumValue(min) SliderRepres.SetMaximumValue(max) SliderRepres.SetValue((min + max) / 2) SliderRepres.SetTitleText("Slider") SliderRepres.GetPoint1Coordinate().SetCoordinateSystemToNormalizedDisplay() SliderRepres.GetPoint1Coordinate().SetValue(0.2, 0.9) SliderRepres.GetPoint2Coordinate().SetCoordinateSystemToNormalizedDisplay() SliderRepres.GetPoint2Coordinate().SetValue(0.8, 0.9) SliderRepres.SetSliderLength(0.02) SliderRepres.SetSliderWidth(0.03) SliderRepres.SetEndCapLength(0.01) SliderRepres.SetEndCapWidth(0.03) SliderRepres.SetTubeWidth(0.005) SliderRepres.SetLabelFormat("%3.0lf") SliderRepres.SetTitleHeight(0.02) SliderRepres.SetLabelHeight(0.02) SliderWidget = vtk.vtkSliderWidget() SliderWidget.SetInteractor(iren) SliderWidget.SetRepresentation(SliderRepres) SliderWidget.KeyPressActivationOff() SliderWidget.SetAnimationModeToAnimate() SliderWidget.SetEnabled(True) SliderWidget.AddObserver("EndInteractionEvent", vtkSliderCallback2) # this causes the pipeline to "execute" renWin.Render() # initialize and start the interactor iren.Initialize() iren.Start() Thanks, Shalini From denismfmcmahon at gmail.com Wed Feb 25 11:22:59 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Wed, 25 Feb 2015 16:22:59 +0000 (UTC) Subject: progress bar References: <981a6a04-b505-4d6c-8c58-935827195a90@googlegroups.com> Message-ID: On Tue, 24 Feb 2015 23:42:42 -0800, Swapnil Pande wrote: > i want to call another tkinter window after completing the progress bar > an n e one help me Try shouting "Oi, Window!" Or show us the code that isn't working (just the bit that isn't working) and explain what it should be doing and what actually happens. Also please tell us what you tried to fix it. For example, you have a problem communicating between output windows after displaying a progress bar, so your example code should: (a) draw two windows (b) display a progress bar in one window (c) Make some sort of call to the second window after the progress bar completes. We only need to see that code. We don't need to see the rest of the code. If my description of what I expect your example code to do doesn't make sense to you, then perhaps it's because your original problem description doesn't make sense to me. ;) -- Denis McMahon, denismfmcmahon at gmail.com From breamoreboy at yahoo.co.uk Wed Feb 25 11:37:25 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 16:37:25 +0000 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On 25/02/2015 06:02, Ian Kelly wrote: > > Is the name of that database program "Microsoft Access" perchance? > Are you referring to the GUI, the underlying database engine, both, or what? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Feb 25 11:40:27 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 16:40:27 +0000 Subject: Python Silent Install In-Reply-To: <54ED872A.7050801@timgolden.me.uk> References: <1424819114.89841.YahooMailMobile@web163806.mail.gq1.yahoo.com> <54ED872A.7050801@timgolden.me.uk> Message-ID: On 25/02/2015 08:26, Tim Golden wrote: > [... re installing with ensurepip disabled ...] > > On 24/02/2015 23:05, Albert-Jan Roskam wrote: >> Personally I find that Python is incomplete without pip and setuptools. > > Of course; that's why the ensurepip was added to the installers. But > there are other ways of installing pip after the event. Including > simply: py -3 -m ensurepip > > TJG > 18 characters, why do the core devs have to make everything so verbose? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From sturla.molden at gmail.com Wed Feb 25 11:44:04 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 25 Feb 2015 17:44:04 +0100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/15 17:04, Peter Otten wrote: > These guys found a bug that is subtler than what most of us have dealt with > in a widely used piece of code originally developed by one of the smarter > members of the python "community". > > I bow my head to them and say thank you. I am not joking about that. It is more the hype this gets that indicates TimSort is already broken today, and even on your cell phone. Sturla From mail at timgolden.me.uk Wed Feb 25 11:49:03 2015 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 25 Feb 2015 16:49:03 +0000 Subject: Python Silent Install In-Reply-To: References: <1424819114.89841.YahooMailMobile@web163806.mail.gq1.yahoo.com> <54ED872A.7050801@timgolden.me.uk> Message-ID: <54EDFCFF.4000705@timgolden.me.uk> On 25/02/2015 16:40, Mark Lawrence wrote: > On 25/02/2015 08:26, Tim Golden wrote: >> [... re installing with ensurepip disabled ...] >> >> On 24/02/2015 23:05, Albert-Jan Roskam wrote: >>> Personally I find that Python is incomplete without pip and setuptools. >> >> Of course; that's why the ensurepip was added to the installers. But >> there are other ways of installing pip after the event. Including >> simply: py -3 -m ensurepip >> >> TJG >> > > 18 characters, why do the core devs have to make everything so verbose? :) > Well of course I was being a bit wordy there. This should be enough: py -mensurepip (assuming that this is your first install and/or that you haven't messed around with your py-loader config). TJG From breamoreboy at yahoo.co.uk Wed Feb 25 11:50:13 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 16:50:13 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/2015 13:58, Sturla Molden wrote: > On 24/02/15 22:34, Roy Smith wrote: >> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >> >> > > This is awful. It is broken for arrays longer than 2**49 elements. With > 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't > see how we can fix this in time. > > Oh yes, and they mention that TimSort is used on billions of devices due > to Android mobile phones. This is clearly very relevant for mobile > phones. Next thing you know your litte Samsung Galaxy with more than > 4096 terabytes breaks down from a stack overflow in TimSort. > > > Sturla > > When I were a lad we only had one bit of data, and we were only able to utilise half of that. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Wed Feb 25 11:55:12 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 16:55:12 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/2015 16:04, Peter Otten wrote: > Sturla Molden wrote: > >> On 24/02/15 22:34, Roy Smith wrote: >>> http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ >>> >> >> This is awful. It is broken for arrays longer than 2**49 elements. With >> 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't >> see how we can fix this in time. >> >> Oh yes, and they mention that TimSort is used on billions of devices due >> to Android mobile phones. This is clearly very relevant for mobile >> phones. Next thing you know your litte Samsung Galaxy with more than >> 4096 terabytes breaks down from a stack overflow in TimSort. > > Yeah, I'm looking forward to see comparable bugs being closed as "cannot > reproduce" by some of the jokers. I suppose that you all had a note > scribbled on the margin of your copy of Arithmetica that this cannot happen > with arrays of lengths seen in practice until 2038 or so. > > These guys found a bug that is subtler than what most of us have dealt with > in a widely used piece of code originally developed by one of the smarter > members of the python "community". > > I bow my head to them and say thank you. > Reading the bug report http://bugs.python.org/issue23515, specifically msg236586, it looks as if the proposed fix was wrong and one of the smarter members of the python community fixed it. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From ckaynor at zindagigames.com Wed Feb 25 11:56:09 2015 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Wed, 25 Feb 2015 08:56:09 -0800 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, Feb 25, 2015 at 8:44 AM, Sturla Molden wrote: > > On 25/02/15 17:04, Peter Otten wrote: > >> These guys found a bug that is subtler than what most of us have dealt with >> in a widely used piece of code originally developed by one of the smarter >> members of the python "community". >> >> I bow my head to them and say thank you. > > > I am not joking about that. It is more the hype this gets that indicates TimSort is already broken today, and even on your cell phone. While the CPython implementation was only broken for arrays of length larger than 2**49, aka, practically not broken, the Java implementation (such as used on Android phones) was broken with arrays of length > 67,108,864 at the time the post was made. While still very large, an array of 67 million elements is well within the realm of possibility today. The Java fixes (so far) have only extended the number out, rather than actually fix the underlying problem - the CPython fixes that were committed implement the full proven fix, so CPython should now be able to handle infinite length arrays (once we can build computers with that much storage...). From ian.g.kelly at gmail.com Wed Feb 25 12:00:14 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Feb 2015 10:00:14 -0700 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Wed, Feb 25, 2015 at 9:37 AM, Mark Lawrence wrote: > On 25/02/2015 06:02, Ian Kelly wrote: >> >> >> Is the name of that database program "Microsoft Access" perchance? >> > > Are you referring to the GUI, the underlying database engine, both, or what? The engine. In theory it supports concurrent access. In practice, it seems that access is frequently blocked by locks generated by some other user who was working on it hours ago and is now out of the office. Not to mention the times when the lock file gets so badly corrupted that the resolution is to just delete it. I haven't used it in some time though, so maybe it's gotten better. From ginolampascione at gmx.com Wed Feb 25 12:07:02 2015 From: ginolampascione at gmx.com (STEVEN DO... EX MORGAN STANLEY) Date: Wed, 25 Feb 2015 09:07:02 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: <6e9f42f8-9802-40b0-a454-dfda15291455@googlegroups.com> -- redacted -- From ginolampascione at gmx.com Wed Feb 25 12:08:28 2015 From: ginolampascione at gmx.com (REVOLUCIOOOOON!!!) Date: Wed, 25 Feb 2015 09:08:28 -0800 (PST) Subject: -- redacted -- In-Reply-To: References: Message-ID: -- redacted -- From breamoreboy at yahoo.co.uk Wed Feb 25 12:16:36 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 17:16:36 +0000 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On 25/02/2015 17:00, Ian Kelly wrote: > On Wed, Feb 25, 2015 at 9:37 AM, Mark Lawrence wrote: >> On 25/02/2015 06:02, Ian Kelly wrote: >>> >>> >>> Is the name of that database program "Microsoft Access" perchance? >>> >> >> Are you referring to the GUI, the underlying database engine, both, or what? > > The engine. In theory it supports concurrent access. In practice, it > seems that access is frequently blocked by locks generated by some > other user who was working on it hours ago and is now out of the > office. Not to mention the times when the lock file gets so badly > corrupted that the resolution is to just delete it. > > I haven't used it in some time though, so maybe it's gotten better. > IIRC the underlying JET engine was replaced by SQL Server years ago. Maybe not the best technlogy in the world, but you'd be hard pushed to do worse than JET :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Wed Feb 25 12:18:47 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 04:18:47 +1100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Thu, Feb 26, 2015 at 3:56 AM, Chris Kaynor wrote: > so > CPython should now be able to handle infinite length arrays (once we > can build computers with that much storage...). Don't we have a Turing Machine buildbot? ChrisA From __peter__ at web.de Wed Feb 25 12:21:25 2015 From: __peter__ at web.de (Peter Otten) Date: Wed, 25 Feb 2015 18:21:25 +0100 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: Mark Lawrence wrote: > Reading the bug report http://bugs.python.org/issue23515, specifically > msg236586, it looks as if the proposed fix was wrong and one of the > smarter members of the python community fixed it. If I understand that correctly Benjamin's modifications don't affect the functionality of the patch. From marfig at gmail.com Wed Feb 25 12:22:25 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Wed, 25 Feb 2015 18:22:25 +0100 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, 25 Feb 2015 17:04:10 +0100, Peter Otten <__peter__ at web.de> wrote: > >These guys found a bug that is subtler than what most of us have dealt with >in a widely used piece of code originally developed by one of the smarter >members of the python "community". > >I bow my head to them and say thank you. And also presented a solution. Thank you for a little of sanity and common sense in this discussion. TYhe levcel of smugness by some of the posters surely can't be a representation of the python community. From rosuav at gmail.com Wed Feb 25 12:22:42 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 04:22:42 +1100 Subject: Are threads bad? - was: Future of Pypy? In-Reply-To: References: <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> <87bnkjenpp.fsf@elektro.pacujo.net> <8761aqamss.fsf@jester.gateway.pace.com> <87zj82bm15.fsf@elektro.pacujo.net> <87pp8ybl14.fsf@elektro.pacujo.net> Message-ID: On Thu, Feb 26, 2015 at 4:16 AM, Mark Lawrence wrote: > IIRC the underlying JET engine was replaced by SQL Server years ago. Maybe > not the best technlogy in the world, but you'd be hard pushed to do worse > than JET :) The way I understood it, MS Access could connect to a variety of database backends (SQL Server, or anything that uses ODBC, or whatever else), but the only inbuilt engine - and therefore the one that you get by default if you aren't running some other server - was MS Jet. If they're incorporating SQL Server into MS Office, that would make it huge... wait, I'm not sure we could tell the difference. But still, that'd be a whopping great slab of database engine. It'd be like Python incorporating PostgreSQL. I've at times said that the Python stdlib ought to include a Postgres *client* (on par with psycopg2), but not the full *server* :) ChrisA From marfig at gmail.com Wed Feb 25 12:29:10 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Wed, 25 Feb 2015 18:29:10 +0100 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, 25 Feb 2015 17:44:04 +0100, Sturla Molden wrote: >I am not joking about that. It is more the hype this gets that indicates >TimSort is already broken today, and even on your cell phone. > But it IS broken.The only hype I'm witnessing is this fantasy created by some that there's a hype around this issue. The team that found the bug produce a non sensationalist and comprehensive report. So it couldn't be them. The other perople discussing the issue are software developers who already know the scope pof this bug from the very beginning of this thread. So I'm not seeing the hype. What I'm seeing is this apparent attempt at reductio ad absurdum to disqualify this bug. From abwilliams72 at gmail.com Wed Feb 25 12:31:29 2015 From: abwilliams72 at gmail.com (Alan Williams) Date: Wed, 25 Feb 2015 09:31:29 -0800 (PST) Subject: Python Captive Portal??? Message-ID: <4856bf29-ab2b-47f2-985d-5911255240eb@googlegroups.com> I was wondering if anyone has experimented with python, memcache and maybe squid to write a simple captive portal managed by a linux box? Concept: Control Center - Linux box running Ubuntu 14.04 w/2 nic's and dhcp-server running on it. NIC's are setup eth0(192.168.0.2), eth1(10.10.10.2). eth0 is the gateway to public internet(connected to DSL Modem) eth1 is serving dhcp addresses to guests through AP's. AP's have static 10. addresses outside of dhcp pool. Guest connect to network and request a webpage. All guests must accept user agreement only(click through page.) and are redirected to their original requested page. Allowed ports are 443 & 80. using python/memcache to manage sessions and redirect to splash page??? Is this possible? What am I missing? From blaxxton at yahoo.com Wed Feb 25 13:19:18 2015 From: blaxxton at yahoo.com (Blaxton) Date: Wed, 25 Feb 2015 18:19:18 +0000 (UTC) Subject: no patch for Python 3.4.2 ? Message-ID: <2075028787.7051571.1424888358342.JavaMail.yahoo@mail.yahoo.com> Hi I don't see any file named patch* in Python 3.4.2 source file.does this mean there is no patch available for this release ? Thanks DB -------------- next part -------------- An HTML attachment was scrubbed... URL: From blaxxton at yahoo.com Wed Feb 25 13:19:18 2015 From: blaxxton at yahoo.com (Blaxton) Date: Wed, 25 Feb 2015 18:19:18 +0000 (UTC) Subject: no patch for Python 3.4.2 ? Message-ID: <2075028787.7051571.1424888358342.JavaMail.yahoo@mail.yahoo.com> Hi I don't see any file named patch* in Python 3.4.2 source file.does this mean there is no patch available for this release ? Thanks DB -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Wed Feb 25 13:41:01 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 25 Feb 2015 19:41:01 +0100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/15 18:22, Mario Figueiredo wrote: > And also presented a solution. Which also was incorrect :-D But now Benjamin Peterson has finally fixed it, it appears: http://bugs.python.org/issue23515 Sturla From jonas at wielicki.name Wed Feb 25 13:46:12 2015 From: jonas at wielicki.name (Jonas Wielicki) Date: Wed, 25 Feb 2015 19:46:12 +0100 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: <54EE1874.3060902@wielicki.name> On 25.02.2015 19:41, Sturla Molden wrote: > On 25/02/15 18:22, Mario Figueiredo wrote: > >> And also presented a solution. > > Which also was incorrect :-D > > But now Benjamin Peterson has finally fixed it, it appears: > > http://bugs.python.org/issue23515 It would be too great if anyone replying to a discussion read the mails before posting ... https://mail.python.org/pipermail/python-list/2015-February/699420.html I support that view, the changes do not change behaviour and merely cosmetic, if I understand that statement correctly regards, jwi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From ian.g.kelly at gmail.com Wed Feb 25 14:42:04 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Feb 2015 12:42:04 -0700 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On Wed, Feb 25, 2015 at 10:21 AM, Peter Otten <__peter__ at web.de> wrote: > Mark Lawrence wrote: > >> Reading the bug report http://bugs.python.org/issue23515, specifically >> msg236586, it looks as if the proposed fix was wrong and one of the >> smarter members of the python community fixed it. > > If I understand that correctly Benjamin's modifications don't affect the > functionality of the patch. Yeah, just looks like some changes for code clarity to me. Py_ssize_t is signed, so the n-1 > 0 test wouldn't break on n = 0. The other changes are pure formatting. From breamoreboy at yahoo.co.uk Wed Feb 25 15:07:06 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 20:07:06 +0000 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25/02/2015 17:22, Mario Figueiredo wrote: > On Wed, 25 Feb 2015 17:04:10 +0100, Peter Otten <__peter__ at web.de> > wrote: >> >> These guys found a bug that is subtler than what most of us have dealt with >> in a widely used piece of code originally developed by one of the smarter >> members of the python "community". >> >> I bow my head to them and say thank you. > > And also presented a solution. > > Thank you for a little of sanity and common sense in this discussion. > TYhe levcel of smugness by some of the posters surely can't be a > representation of the python community. > I don't believe that any technical community with exactly 4,750 open issues on its bug tracker regards itself as smug. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Wed Feb 25 15:21:13 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 07:21:13 +1100 Subject: no patch for Python 3.4.2 ? In-Reply-To: <2075028787.7051571.1424888358342.JavaMail.yahoo@mail.yahoo.com> References: <2075028787.7051571.1424888358342.JavaMail.yahoo@mail.yahoo.com> Message-ID: On Thu, Feb 26, 2015 at 5:19 AM, Blaxton wrote: > I don't see any file named patch* in Python 3.4.2 source file. > does this mean there is no patch available for this release ? Not sure what you mean. Are you looking for a single massive patch which updates your 3.4.1 source code to 3.4.2? Or are you looking for a patch which updates your binary? The first isn't necessary; you simply get all the sources for 3.4.2, and that way, you don't have to worry about exactly _what_ you're upgrading from. The second is virtually impossible, because we can't know what your binary is. It's much safer to compile fresh from source. If you're using a package manager to install Python, try to use the package manager to upgrade, too. That's the cleanest way to get the latest - if it's possible. If it isn't, building from source is usually the best option on Linux; on Windows, you'll probably find that it's easier to download the .msi installer and use that. ChrisA From breamoreboy at yahoo.co.uk Wed Feb 25 15:45:44 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 25 Feb 2015 20:45:44 +0000 Subject: Python Worst Practices Message-ID: http://www.slideshare.net/pydanny/python-worst-practices Any that should be added to this list? Any that be removed as not that bad? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From motoom at xs4all.nl Wed Feb 25 15:58:48 2015 From: motoom at xs4all.nl (Michiel Overtoom) Date: Wed, 25 Feb 2015 21:58:48 +0100 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <65553271-F7BD-4717-AF82-9AE2B591A532@xs4all.nl> On Feb 25, 2015, at 21:45, Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices I agree with you that Python lambdas have little use beyond the most trivial use cases. For the non-trivial cases, I like to define a named function which does the job. And also provides documentation, just by virtue of being named (and having a docstring). I also tend to do this in JavaScript code, which also can benefit from this. Greetings, -- "You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes From tjreedy at udel.edu Wed Feb 25 16:08:04 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Feb 2015 16:08:04 -0500 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 2/25/2015 10:23 AM, Zachary Ware wrote: > Just to be clear, this has already been fixed, and the fix will be > released in Python 2.7.10, 3.4.4, and 3.5.0. I think the important effect of the fix is to encourage automatic code verification both by the group that found this bug and by and others. Future verifier developers can check their code by trying to verify the fixed code. The C half of the CPython code base has been improved by responding to issues raised by the Coverity checker. I hope we can get free code reviews both of C and Python code from others. -- Terry Jan Reedy From rosuav at gmail.com Wed Feb 25 16:24:37 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 08:24:37 +1100 Subject: Python Worst Practices In-Reply-To: References: Message-ID: On Thu, Feb 26, 2015 at 7:45 AM, Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? Remove the complaint about id. It's an extremely useful variable name, and you hardly ever need the function. The only reason to avoid it is so you get an immediate NameError if you haven't assigned to id, and the alternative is pretty easily debuggable. I agree that "zip" can easily be replaced with "zip_code" or "postal_code"; the need to avoid "object" is almost completely unnecessary for me as I'd always abbreviate it to "obj" anyway, but it doesn't hurt to say it. (Flipping the booleans makes no sense to me. When would 0 mean true and 1 mean false? Isn't it much more likely that, for instance, 0 means success and nonzero means error (and maybe there's just one error state, so 1 means failure)?) I'd take the one about properties a bit further. Try to avoid using settable properties *at all*, until you've proven that you absolutely do need them. Just let the calling code directly mutate your attributes! I'd also strengthen the point about "except: pass". He says "Arguably better to not have this in your code". I don't think there's any "arguably" about it - silently swallowing *any* exception is just wrong. (Incidentally, the advent of chained exceptions may mean that some of the code can be simplified - no need to explicitly retain the previous exception when raising a custom one.) But I take issue with the general ban on lambda. He's showing an atrocious use of lambda: 1) Assigning a lambda function to a simple name is pointless. If you're not making use of the fact that lambda is an expression, use def. 2) If you need to fiddle around with dunder methods to do what you want, then you definitely shouldn't be doing this as a lambda function. You're fighting against the language by wrapping your logic up until it can be an expression, only to ignore the value of that expression. 3) And if you need a nested function call to implement a local variable, because your dunder method calling is getting ridiculously long, then duh, use def! I could come up with an equally atrocious mis-use of list comprehensions. Does that mean that a list comp is "Python worst practice", and we should unroll them all into assignment, loop, append? No. Certainly not. [1] It just proves that the feature can be misused. Hah. Talking about old-style classes. Okay, here's another Python best practice: Use Python 3 if you can. :) But if you can't, then I definitely agree that you should subclass object, put parens around your print arguments, use "as" when capturing exceptions, all those little things that will make your code that bit more 2/3 compatible without making it any worse Py2 code. Additional Python worst practice, although I don't see it all that often: Creating classes for anything and everything. In a lot of cases, Python code works just fine at top-level (eg module-level functions), and those levels of indirection don't make the code any easier to read. ChrisA [1] Sooooooo..... Off we go to the Gaiety, for that's the place for me! [2] [2] But you probably don't know that song, and I can't find it on Youtube. "Johnny at the Gaiety", by George Grossmith, back in the late 19th century. From python at mrabarnett.plus.com Wed Feb 25 16:28:10 2015 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 25 Feb 2015 21:28:10 +0000 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <54EE3E6A.5010204@mrabarnett.plus.com> On 2015-02-25 20:45, Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? > We don't have numeric ZIP codes in the UK, but the entire world has numeric telephone numbers, so that might be a better example of numbers that aren't really numbers. Numeric dates can be ambiguous: dd/mm/yyyy or mm/dd/yyyy? The ISO standard is clearer: yyyy-mm-dd. Handling text: "Unicode sandwich". UTF-8 is better than legacy encodings. From joel.goldstick at gmail.com Wed Feb 25 17:59:10 2015 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 25 Feb 2015 17:59:10 -0500 Subject: Python Worst Practices In-Reply-To: <54EE3E6A.5010204@mrabarnett.plus.com> References: <54EE3E6A.5010204@mrabarnett.plus.com> Message-ID: On Wed, Feb 25, 2015 at 4:28 PM, MRAB wrote: > On 2015-02-25 20:45, Mark Lawrence wrote: >> >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that >> bad? >> > We don't have numeric ZIP codes in the UK, but the entire world has > numeric telephone numbers, so that might be a better example of numbers > that aren't really numbers. US zip codes get messed up with ints because many have a leading zero. I use strings > > Numeric dates can be ambiguous: dd/mm/yyyy or mm/dd/yyyy? The ISO > standard is clearer: yyyy-mm-dd. > > Handling text: "Unicode sandwich". > > UTF-8 is better than legacy encodings. > > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com From invalid at invalid.invalid Wed Feb 25 18:00:46 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 25 Feb 2015 23:00:46 +0000 (UTC) Subject: Python Worst Practices References: Message-ID: On 2015-02-25, MRAB wrote: > On 2015-02-25 20:45, Mark Lawrence wrote: >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that bad? > > We don't have numeric ZIP codes in the UK, but the entire world has > numeric telephone numbers, That's probably true now, but hasn't always been the case (at least in the US). If you're dealing with historical data, you might need to include the city and exchange name in addition to the 4/5-digit line number. In _most_ multi-exchange service areas, those exchange names have always mapped to a 2-digit number, but I believe there were exceptions. When I were a wee kid in the 60's my phone number was "Cedar 2 ". By then everybody had all-numeric dialing and "Cedar" translated to "23", but it was still common to see phone numbers printed using the exchange names. > so that might be a better example of numbers that aren't really > numbers. -- Grant Edwards grant.b.edwards Yow! Am I elected yet? at gmail.com From steve+comp.lang.python at pearwood.info Wed Feb 25 18:13:49 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 26 Feb 2015 10:13:49 +1100 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: <54ee572e$0$13007$c3e8da3$5496439d@news.astraweb.com> Sturla Molden wrote: > On 25/02/15 17:04, Peter Otten wrote: > >> These guys found a bug that is subtler than what most of us have dealt >> with in a widely used piece of code originally developed by one of the >> smarter members of the python "community". >> >> I bow my head to them and say thank you. > > I am not joking about that. It is more the hype this gets that indicates > TimSort is already broken today, and even on your cell phone. TimSort is an algorithm, and it is not broken. The algorithm is correct. Any specific implementation of that algorithm may or may not be correct. As Zachary has pointed out, the implementation used in CPython has already been fixed, for versions 2.7.10, 3.4.4, and 3.5.0, and is now correct (as far as anyone can tell). Other implementations -- including Java and Android phones -- are not our responsibility or concern, except so far as they may effect Jython or other Python code. -- Steven From python at mrabarnett.plus.com Wed Feb 25 18:34:29 2015 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 25 Feb 2015 23:34:29 +0000 Subject: Python Worst Practices In-Reply-To: References: <54EE3E6A.5010204@mrabarnett.plus.com> Message-ID: <54EE5C05.3060005@mrabarnett.plus.com> On 2015-02-25 22:59, Joel Goldstick wrote: > On Wed, Feb 25, 2015 at 4:28 PM, MRAB wrote: > > On 2015-02-25 20:45, Mark Lawrence wrote: > >> > >> http://www.slideshare.net/pydanny/python-worst-practices > >> > >> Any that should be added to this list? Any that be removed as not that > >> bad? > >> > > We don't have numeric ZIP codes in the UK, but the entire world has > > numeric telephone numbers, so that might be a better example of numbers > > that aren't really numbers. > > US zip codes get messed up with ints because many have a leading zero. > I use strings Telephone numbers can also start with zero. > > > > Numeric dates can be ambiguous: dd/mm/yyyy or mm/dd/yyyy? The ISO > > standard is clearer: yyyy-mm-dd. > > > > Handling text: "Unicode sandwich". > > > > UTF-8 is better than legacy encodings. > > From ben+python at benfinney.id.au Wed Feb 25 18:45:04 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 10:45:04 +1100 Subject: Python Worst Practices References: Message-ID: <85fv9t5zen.fsf@benfinney.id.au> Mark Lawrence writes: > http://www.slideshare.net/pydanny/python-worst-practices Slideshare requires runing untrusted code in my browser, just to view the slides. Can someone direct us to a URL where the document can be downloaded anonymously for offline viewing? -- \ ?What is needed is not the will to believe but the will to find | `\ out, which is the exact opposite.? ?Bertrand Russell, _Free | _o__) Thought and Official Propaganda_, 1928 | Ben Finney From ben+python at benfinney.id.au Wed Feb 25 18:48:21 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 10:48:21 +1100 Subject: Python Worst Practices References: Message-ID: <85bnkh5z96.fsf@benfinney.id.au> Chris Angelico writes: > (Flipping the booleans makes no sense to me. When would 0 mean true > and 1 mean false? Isn't it much more likely that, for instance, 0 > means success and nonzero means error (and maybe there's just one > error state, so 1 means failure)?) You've answered your question, I believe. In some contexts ? such as Unix shell ? 0 ? success ? true, and non-0 ? failure ? false. The Unix commands ?true? and ?false? follow that convention . -- \ ?Know what I hate most? Rhetorical questions.? ?Henry N. Camp | `\ | _o__) | Ben Finney From steve+comp.lang.python at pearwood.info Wed Feb 25 18:54:22 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 26 Feb 2015 10:54:22 +1100 Subject: Python Worst Practices References: Message-ID: <54ee60af$0$13001$c3e8da3$5496439d@news.astraweb.com> Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that > bad? I have no idea about Python worst practices, but I think using some sort of Powerpoint slide is surely one of the worst practices in general. In my browser, I see the title "Python Worst Practices". There's a Next Slide button, but clicking it does nothing. Down below, there is what appears to be a transcript, but it is full of extraneous and unnecessary fluff (nearly every line has a leading "Daniel Greenfeld @pydanny") and the formatting is destroyed. Worst practices: - Using assert for error checking, instead of explicit tests. - Overuse of regexes. If the only tool you have is the regex hammer, everything looks like a nail. - Underuse of regexes: failing to use a regex when one is called for is not quite as bad as the regex hammer, but still. - Python is not Java. http://dirtsimple.org/2004/12/python-is-not-java.html? http://dirtsimple.org/2004/12/java-is-not-python-either.html Most worst practices are independent of the language. Regardless of whether they are writing Python, C or Ocaml, too many programmers make these deadly programming sins: - Too much coupling. Possibly the Mother Of All Sins, many other sins are variations of this: spaghetti code, ravioli code, global variables, etc. - Premature optimization. Which usually means, optimizing without measurement. If you just *assume* this code is "faster", chances are excellent that it is actually slower. - Overuse of 1-character or excessively generic variable names. - The other extreme: extremely_long_variable_names_for_no_good_reason. - Variable names (including functions and classes!) that aren't self-explanatory. - Misusing Hungarian notation. Or failing to use it when appropriate. http://www.joelonsoftware.com/articles/Wrong.html - Violating the Rule of Demeter: don't talk to the dog's leg, talk to the dog. Or another way to put it: don't let the paper boy reach into your pocket for money. And very possibly the worst practice of all: - Failing to understand when it is, and isn't, appropriate to break the rules and do what would otherwise be a bad practice. -- Steven From rosuav at gmail.com Wed Feb 25 18:57:20 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 10:57:20 +1100 Subject: Python Worst Practices In-Reply-To: <85bnkh5z96.fsf@benfinney.id.au> References: <85bnkh5z96.fsf@benfinney.id.au> Message-ID: On Thu, Feb 26, 2015 at 10:48 AM, Ben Finney wrote: > Chris Angelico writes: > >> (Flipping the booleans makes no sense to me. When would 0 mean true >> and 1 mean false? Isn't it much more likely that, for instance, 0 >> means success and nonzero means error (and maybe there's just one >> error state, so 1 means failure)?) > > You've answered your question, I believe. In some contexts ? such as > Unix shell ? 0 ? success ? true, and non-0 ? failure ? false. > > The Unix commands ?true? and ?false? follow that convention > . Yes, but my point is: You shouldn't need to rebind those names (or have names "true" and "false" for 0 and 1). Instead, use "success" and "failure". That is to say: The function doesn't return a boolean for whether it did something or not, it returns an error code (or 0 if no error). I agree with the presentation, but have never seen code that does this; it's like saying "Python worst practice: Defining your integer constants using Shakespeare syntax. Python best practice: Defining your integer constants as decimal or hexadecimal digits.", in that nobody would ever do the worst-practice side of it. But maybe I've lived a sheltered life... ChrisA From ben+python at benfinney.id.au Wed Feb 25 19:11:24 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 11:11:24 +1100 Subject: Python Worst Practices References: <85bnkh5z96.fsf@benfinney.id.au> Message-ID: <85zj814jmb.fsf@benfinney.id.au> Chris Angelico writes: > On Thu, Feb 26, 2015 at 10:48 AM, Ben Finney wrote: > > Chris Angelico writes: > > > >> (Flipping the booleans makes no sense to me. When would 0 mean true > >> and 1 mean false? [?]) > > > > The Unix commands ?true? and ?false? follow that convention > > . > > Yes, but my point is: You shouldn't need to rebind those names (or > have names "true" and "false" for 0 and 1). That's not what you asked, though. You asked ?When would 0 mean true and 1 mean false?? My answer: in all Unix shell contexts. > Instead, use "success" and "failure". You'd better borrow the time machine and tell the creators of Unix. The meme is already established for decades now. -- \ ?Firmness in decision is often merely a form of stupidity. It | `\ indicates an inability to think the same thing out twice.? | _o__) ?Henry L. Mencken | Ben Finney From roy at panix.com Wed Feb 25 19:12:51 2015 From: roy at panix.com (Roy Smith) Date: Wed, 25 Feb 2015 19:12:51 -0500 Subject: Bug in timsort!? References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: In article , Sturla Molden wrote: > On 24/02/15 22:34, Roy Smith wrote: > > http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm > > -is-broken-and-how-to-fix-it/ > > > > This is awful. It is broken for arrays longer than 2**49 elements. With > 8 bytes per PyObject* pointer this is >4096 terabytes of RAM. I don't > see how we can fix this in time. > > Oh yes, and they mention that TimSort is used on billions of devices due > to Android mobile phones. This is clearly very relevant for mobile > phones. Next thing you know your litte Samsung Galaxy with more than > 4096 terabytes breaks down from a stack overflow in TimSort. Keep in mind that the phone I cary around today has more memory (and probably more CPU) than the biggest supercomputers that existed when I was in college. From rosuav at gmail.com Wed Feb 25 19:26:40 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 11:26:40 +1100 Subject: Python Worst Practices In-Reply-To: <54ee60af$0$13001$c3e8da3$5496439d@news.astraweb.com> References: <54ee60af$0$13001$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 26, 2015 at 10:54 AM, Steven D'Aprano wrote: > - Violating the Rule of Demeter: don't talk to the dog's leg, talk to > the dog. Or another way to put it: don't let the paper boy reach > into your pocket for money. I'd call that code smell, rather than an automatic worst practice. Suppose this: class Shelf: def __init__(self): self.items = [] # Empty shelf bookshelf = Shelf() bookshelf.items.append(book) To enforce Demeter, you'd have to add a bunch of methods to the Shelf whose sole purpose is to pass word along to the items list. Sure, it makes some design sense to say "Add this book to the shelf" rather than "Add this book to the items on the shelf", but all those lines of code are potential bugs, and if you have to reimplement huge slabs of functionality, that too is code smell. So there are times when it's correct to reach into another object. But the times to use two dots are much rarer than the times to use one dot (the paper boy shouldn't reach into your pocket for money, but ThinkGeek has your credit card number on file so you can order more conveniently), and I can't think of any example off-hand where you would want more than three dots. ChrisA From rosuav at gmail.com Wed Feb 25 19:37:31 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 11:37:31 +1100 Subject: Python Worst Practices In-Reply-To: <85zj814jmb.fsf@benfinney.id.au> References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> Message-ID: On Thu, Feb 26, 2015 at 11:11 AM, Ben Finney wrote: > Chris Angelico writes: > >> On Thu, Feb 26, 2015 at 10:48 AM, Ben Finney wrote: >> > Chris Angelico writes: >> > >> >> (Flipping the booleans makes no sense to me. When would 0 mean true >> >> and 1 mean false? [?]) >> > >> > The Unix commands ?true? and ?false? follow that convention >> > . >> >> Yes, but my point is: You shouldn't need to rebind those names (or >> have names "true" and "false" for 0 and 1). > > That's not what you asked, though. You asked ?When would 0 mean true and > 1 mean false?? My answer: in all Unix shell contexts. My bad. I was talking in a context of Python programming, specifically with APIs where you would use some kind of true/false flag as either a function parameter or a return value. There are plenty of contexts where you want a success/failure response, but you can name your constants "success" and "failure" for that, rather than "true" and "false". >> Instead, use "success" and "failure". > > You'd better borrow the time machine and tell the creators of Unix. The > meme is already established for decades now. IMO the whole system of boolean logic in shell scripts is a massive pile of hacks. In its purest sense, you have "do this, and if it succeeds, do that": cp source destination && rm source which makes reasonable sense with an "and" operator - do this, and do this - and the converse makes similar sense with an "or" operator: cp source destination || echo Oopsie On the assumption that a nonzero return code from cp means it didn't copy the file, this reads as "copy the file or emit that message", which is exactly right. Nice. But that's stretching the notion of "and" and "or" a little, to the point where booleans and comparisons stop making so much sense: [ -f /usr/local/bin/python ] || sudo make install Is that a success/failure, or is it an if-then? Kinda neither, kinda both, kinda one built on the other built on the other. cp source destination || true Now this makes no sense outside of its very specific purpose. "Or true"? When would you use that? What it *actually* means is "attempt this copy, and if it fails, pretend it succeeded" (presumably because you have something aborting on error, like "set -e"). It's not boolean logic, and it's not really an alternate command for the failure case. The sh and bash system of logic works, there's no doubt about it. It takes "practicality beats purity" a very long way, and makes something where a fairly simple construct ("||" just means "if nonzero error code, execute this, otherwise don't") can do three or four barely-related jobs, which makes the language more compact. But none of this means that Python should, in any way, imitate it. ChrisA From ben+python at benfinney.id.au Wed Feb 25 20:05:12 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 12:05:12 +1100 Subject: Python Worst Practices References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> Message-ID: <85r3td4h4n.fsf@benfinney.id.au> Chris Angelico writes: > IMO the whole system of boolean logic in shell scripts is a massive > pile of hacks. Agreed. It bears all the hallmarks of a system which has been extended to become a complete programming language only with extreme reluctance on its part. I continue to be impressed by how capable and powerful Unix shell is as a full programming language. Especially it is sorely missed on other OSes which lack a capable shell. But it could never be called ?elegant?. > The sh and bash system of logic works, there's no doubt about it. [?] > But none of this means that Python should, in any way, imitate it. Quite so. -- \ ?Airports are ugly. Some are very ugly. Some attain a degree of | `\ ugliness that can only be the result of a special effort.? | _o__) ?Douglas Adams, _The Long Dark Tea-Time of the Soul_, 1988 | Ben Finney From breamoreboy at yahoo.co.uk Wed Feb 25 20:44:30 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 26 Feb 2015 01:44:30 +0000 Subject: Python Worst Practices In-Reply-To: References: Message-ID: On 25/02/2015 20:45, Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that > bad? > Throwing in my own, how about built-in functions should not use "object" as the one and only argument, and a keyword argument at that. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From john_ladasky at sbcglobal.net Wed Feb 25 21:35:18 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Wed, 25 Feb 2015 18:35:18 -0800 (PST) Subject: Parallelization of Python on GPU? Message-ID: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> I've been working with machine learning for a while. Many of the standard packages (e.g., scikit-learn) have fitting algorithms which run in single threads. These algorithms are not themselves parallelized. Perhaps, due to their unique mathematical requirements, they cannot be paralleized. When one is investigating several potential models of one's data with various settings for free parameters, it is still sometimes possible to speed things up. On a modern machine, one can use Python's multiprocessing.Pool to run separate instances of scikit-learn fits. I am currently using ten of the twelve 3.3 GHz CPU cores on my machine to do just that. And I can still browse the web with no observable lag. :^) Still, I'm waiting hours for jobs to finish. Support vector regression fitting is hard. What I would REALLY like to do is to take advantage of my GPU. My NVidia graphics card has 1152 cores and a 1.0 GHz clock. I wouldn't mind borrowing a few hundred of those GPU cores at a time, and see what they can do. In theory, I calculate that I can speed up the job by another five-fold. The trick is that each process would need to run some PYTHON code, not CUDA or OpenCL. The child process code isn't particularly fancy. (I should, for example, be able to switch that portion of my code to static typing.) What is the most effective way to accomplish this task? I came across a reference to a package called "Urutu" which may be what I need, however it doesn't look like it is widely supported. I would love it if the Python developers themselves added the ability to spawn GPU processes to the Multiprocessing module! Thanks for any advice and comments. From torriem at gmail.com Wed Feb 25 21:45:06 2015 From: torriem at gmail.com (Michael Torrie) Date: Wed, 25 Feb 2015 19:45:06 -0700 Subject: Python Worst Practices In-Reply-To: <85fv9t5zen.fsf@benfinney.id.au> References: <85fv9t5zen.fsf@benfinney.id.au> Message-ID: <54EE88B2.1020008@gmail.com> On 02/25/2015 04:45 PM, Ben Finney wrote: > Mark Lawrence writes: > >> http://www.slideshare.net/pydanny/python-worst-practices > > Slideshare requires runing untrusted code in my browser, just to view > the slides. > > Can someone direct us to a URL where the document can be downloaded > anonymously for offline viewing? Ahh, so you're referring to the Javascript? From steve+comp.lang.python at pearwood.info Wed Feb 25 21:57:25 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 26 Feb 2015 13:57:25 +1100 Subject: Python Worst Practices References: Message-ID: <54ee8b96$0$11124$c3e8da3@news.astraweb.com> Mark Lawrence wrote: > On 25/02/2015 20:45, Mark Lawrence wrote: >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that >> bad? >> > > Throwing in my own, how about built-in functions should not use "object" > as the one and only argument, and a keyword argument at that. Which built-in function is that? -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 25 22:02:56 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 26 Feb 2015 14:02:56 +1100 Subject: Parallelization of Python on GPU? References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> John Ladasky wrote: > What I would REALLY like to do is to take advantage of my GPU. I can't help you with that, but I would like to point out that GPUs typically don't support IEE-754 maths, which means that while they are likely significantly faster, they're also likely significantly less accurate. Any any two different brands/models of GPU are likely to give different results. (Possibly not *very* different, but considering the mess that floating point maths was prior to IEEE-754, possibly *very* different.) Personally, I wouldn't trust GPU floating point for serious work. Maybe for quick and dirty exploration of the data, but I'd then want to repeat any calculations using the main CPU before using the numbers anywhere :-) -- Steve From ethan at stoneleaf.us Wed Feb 25 22:03:29 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 25 Feb 2015 19:03:29 -0800 Subject: Parallelization of Python on GPU? In-Reply-To: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: <54EE8D01.8040406@stoneleaf.us> On 02/25/2015 06:35 PM, John Ladasky wrote: > What I would REALLY like to do is to take advantage of my GPU. My NVidia graphics > card has 1152 cores and a 1.0 GHz clock. I wouldn't mind borrowing a few hundred > of those GPU cores at a time, and see what they can do. In theory, I calculate > that I can speed up the job by another five-fold. Only free for academic use: https://developer.nvidia.com/how-to-cuda-python unsure, but looks like free to use: http://mathema.tician.de/software/pycuda/ and, of course, the StackOverflow question: http://stackoverflow.com/q/5957554/208880 -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From davea at davea.name Wed Feb 25 22:05:07 2015 From: davea at davea.name (Dave Angel) Date: Wed, 25 Feb 2015 22:05:07 -0500 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <54EE8D63.2050208@davea.name> On 02/25/2015 08:44 PM, Mark Lawrence wrote: > On 25/02/2015 20:45, Mark Lawrence wrote: >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that >> bad? >> > > Throwing in my own, how about built-in functions should not use "object" > as the one and only argument, and a keyword argument at that. > def marry(object = False)... if anybody has any cause to object, let him speak now... -- DaveA From ethan at stoneleaf.us Wed Feb 25 22:05:51 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 25 Feb 2015 19:05:51 -0800 Subject: Parallelization of Python on GPU? In-Reply-To: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: <54EE8D8F.4050803@stoneleaf.us> Oh, and this one: http://www.cs.toronto.edu/~tijmen/gnumpy.html -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From rosuav at gmail.com Wed Feb 25 22:17:12 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 14:17:12 +1100 Subject: Python Worst Practices In-Reply-To: <54EE8D63.2050208@davea.name> References: <54EE8D63.2050208@davea.name> Message-ID: On Thu, Feb 26, 2015 at 2:05 PM, Dave Angel wrote: > On 02/25/2015 08:44 PM, Mark Lawrence wrote: >> >> On 25/02/2015 20:45, Mark Lawrence wrote: >>> >>> http://www.slideshare.net/pydanny/python-worst-practices >>> >>> Any that should be added to this list? Any that be removed as not that >>> bad? >>> >> >> Throwing in my own, how about built-in functions should not use "object" >> as the one and only argument, and a keyword argument at that. >> > > def marry(object = False)... > > > if anybody has any cause to object, let him speak now... Python. Python is what bwings us togethar, today. And def, twue def, will make your argument into a keyword argument that will fowwow you forevah and evah. So tweasure your keyword arguments... ChrisA From john_ladasky at sbcglobal.net Wed Feb 25 23:01:13 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Wed, 25 Feb 2015 20:01:13 -0800 (PST) Subject: Parallelization of Python on GPU? In-Reply-To: <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> Message-ID: <459a9366-19ef-4f98-9087-e50430a8655e@googlegroups.com> On Wednesday, February 25, 2015 at 7:03:23 PM UTC-8, Steven D'Aprano wrote: > I would like to point out that GPUs > typically don't support IEE-754 maths, which means that while they are > likely significantly faster, they're also likely significantly less > accurate. Historically, that has been true. According to this document... https://developer.nvidia.com/sites/default/files/akamai/cuda/files/NVIDIA-CUDA-Floating-Point.pdf ...NVidia's GPU cards which implement "compute capability" versions 2.0 and higher are IEEE-754 compliant, both for single- and double-precision floating point operations. The current "compute capability" version is 5.2, so there are several generations of GPU hardware out there by now which should give satisfactory floating-point results. From ben+python at benfinney.id.au Wed Feb 25 23:08:25 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 15:08:25 +1100 Subject: Python Worst Practices References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> Message-ID: <85d24x48na.fsf@benfinney.id.au> Michael Torrie writes: > On 02/25/2015 04:45 PM, Ben Finney wrote: > > Can someone direct us to a URL where the document can be downloaded > > anonymously for offline viewing? > > Ahh, so you're referring to the Javascript? I'm referring to the fact that a presentation document at Slideshare is not viewable nor downloadable, unless I identify myself or run a browser app. Both of which have no technical justification; a PDF presentation document would do just as well and avoid those problems. When directing people to view one's presentation document, please make sure the document is available via a simple download, not a needless user-registration process. -- \ ?Philosophy is questions that may never be answered. Religion | `\ is answers that may never be questioned.? ?anonymous | _o__) | Ben Finney From rosuav at gmail.com Wed Feb 25 23:17:13 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 15:17:13 +1100 Subject: Python Worst Practices In-Reply-To: <85d24x48na.fsf@benfinney.id.au> References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> Message-ID: On Thu, Feb 26, 2015 at 3:08 PM, Ben Finney wrote: > Both of which have no technical justification; a PDF presentation > document would do just as well and avoid those problems. Probably doesn't even need PDF. The slides are approximately as complex as those in Ned Batchelder's "Pragmatic Unicode" presentation, which is represented entirely in text. Maybe there's some slight additional complications, but not many. I'd really like to see a lot more presentations done in pure text. Scrap the graphics, scrap the fancy transitions, they aren't helping you make your case anyway. Just give me the content. Tell me what you're saying. I don't care about the fluff. ChrisA From ben+python at benfinney.id.au Wed Feb 25 23:44:18 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 15:44:18 +1100 Subject: Python Worst Practices References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> Message-ID: <858ufl46zh.fsf@benfinney.id.au> Chris Angelico writes: > I'd really like to see a lot more presentations done in pure text. Maybe so. My request at the moment, though, is not for people to change what's on their slides; rather, if they want people to retrieve them, the slides should be downloadable easily (i.e. without a web app, without a registration to some specific site). -- \ ?I thought I'd begin by reading a poem by Shakespeare, but then | `\ I thought ?Why should I? He never reads any of mine.?? ?Spike | _o__) Milligan | Ben Finney From rosuav at gmail.com Wed Feb 25 23:47:33 2015 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Feb 2015 15:47:33 +1100 Subject: Python Worst Practices In-Reply-To: <858ufl46zh.fsf@benfinney.id.au> References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> <858ufl46zh.fsf@benfinney.id.au> Message-ID: On Thu, Feb 26, 2015 at 3:44 PM, Ben Finney wrote: > Chris Angelico writes: > >> I'd really like to see a lot more presentations done in pure text. > > Maybe so. My request at the moment, though, is not for people to change > what's on their slides; rather, if they want people to retrieve them, > the slides should be downloadable easily (i.e. without a web app, > without a registration to some specific site). Yeah, my "really like to see" is more about decisions made when the slides are created. For this, PDF or PNG images would probably be the most convenient. ChrisA From torriem at gmail.com Wed Feb 25 23:50:10 2015 From: torriem at gmail.com (Michael Torrie) Date: Wed, 25 Feb 2015 21:50:10 -0700 Subject: Python Worst Practices In-Reply-To: <85d24x48na.fsf@benfinney.id.au> References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> Message-ID: <54EEA602.50003@gmail.com> On 02/25/2015 09:08 PM, Ben Finney wrote: > Michael Torrie writes: > >> On 02/25/2015 04:45 PM, Ben Finney wrote: >>> Can someone direct us to a URL where the document can be downloaded >>> anonymously for offline viewing? >> >> Ahh, so you're referring to the Javascript? > > I'm referring to the fact that a presentation document at Slideshare is > not viewable nor downloadable, unless I identify myself or run a browser > app. I'm curious. What browser app are you referring to here? Seems like the presentation is pure html5 with a bit a javascript. Is this what you mean? I didn't require any special apps to view the slideshow. I certainly didn't "identify" myself any more than I would for any other web site (exposing my IP address). Anyway just trying to understand exactly what you mean. I'm not sure if you're a noscript person, or if I've missed something. I agree about the PDF and offline viewing. From ian.g.kelly at gmail.com Wed Feb 25 23:52:26 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Feb 2015 21:52:26 -0700 Subject: Python Worst Practices In-Reply-To: References: Message-ID: On Wed, Feb 25, 2015 at 1:45 PM, Mark Lawrence wrote: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? Using XML for configuration is a good example of a worst practice, but using Python instead isn't best practice. There are good arguments that a configuration language shouldn't be Turing-complete. See for instance this blog post: http://taint.org/2011/02/18/001527a.html The problem with XML is just that it's not very human-readable; it doesn't need all the extra power of Python. This may be something that's hard to appreciate until you've actually experienced an outage because somebody checked in a configuration change that parsed and passed tests but nonetheless was invalid due to configuration that couldn't be evaluated until runtime. A better choice for configuration would be something that is both easily readable and *simple* such as JSON or YAML, or an appropriate DSL. From ben+python at benfinney.id.au Thu Feb 26 00:21:59 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 26 Feb 2015 16:21:59 +1100 Subject: Python Worst Practices References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> <54EEA602.50003@gmail.com> Message-ID: <854mq9458o.fsf@benfinney.id.au> Michael Torrie writes: > I'm curious. What browser app are you referring to here? Seems like the > presentation is pure html5 with a bit a javascript. Is this what you > mean? Yes. This is a document we're talking about, with many pages. I don't want to sit navigating in my browser to read it, when it's certainly in a form that could just be presented for download and viewed in a program of my choice, while offline. > I certainly didn't "identify" myself any more than I would for any > other web site (exposing my IP address). Try downloading the document, without registering or logging in ? neither of which should be needed for a document supposedly for public view. -- \ ?I went to a museum where all the artwork was done by children. | `\ They had all the paintings up on refrigerators.? ?Steven Wright | _o__) | Ben Finney From greg.ewing at canterbury.ac.nz Thu Feb 26 00:36:17 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 26 Feb 2015 18:36:17 +1300 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: Cem Karan wrote: > I think I see what you're talking about now. Does WeakMethod > (https://docs.python.org/3/library/weakref.html#weakref.WeakMethod) solve > this problem? Yes, that looks like it would work. -- Greg From torriem at gmail.com Thu Feb 26 00:41:56 2015 From: torriem at gmail.com (Michael Torrie) Date: Wed, 25 Feb 2015 22:41:56 -0700 Subject: Python Worst Practices In-Reply-To: <854mq9458o.fsf@benfinney.id.au> References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> <54EEA602.50003@gmail.com> <854mq9458o.fsf@benfinney.id.au> Message-ID: <54EEB224.3060701@gmail.com> On 02/25/2015 10:21 PM, Ben Finney wrote: > document we're talking about, with many pages. I don't > want to sit navigating in my browser to read it, when it's certainly in > a form that could just be presented for download and viewed in a program > of my choice, while offline. > >> > I certainly didn't "identify" myself any more than I would for any >> > other web site (exposing my IP address). > Try downloading the document, without registering or logging in ? > neither of which should be needed for a document supposedly for public > view. Okay I understand. Yes I agree with you about the download completely. From ian.g.kelly at gmail.com Thu Feb 26 00:51:42 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Feb 2015 22:51:42 -0700 Subject: Bug in timsort!? In-Reply-To: <54ee572e$0$13007$c3e8da3$5496439d@news.astraweb.com> References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> <54ee572e$0$13007$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 25, 2015 at 4:13 PM, Steven D'Aprano wrote: > TimSort is an algorithm, and it is not broken. The algorithm is correct. The algorithm asserted an invariant but failed to actually establish it. That sounds broken to me. From john_ladasky at sbcglobal.net Thu Feb 26 00:53:02 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Wed, 25 Feb 2015 21:53:02 -0800 (PST) Subject: Parallelization of Python on GPU? In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: Thanks for the various links, Ethan. I have encountered PyCUDA before, but not the other options. So far, I'm not seeing code examples which appear to do what I would like, which is simply to farm out one Python process to one GPU core. The examples all appear to parallelize array operations. I know, that's the easier way to break up a task. I may have to bite the bullet and learn how to use this: http://mklab.iti.gr/project/GPU-LIBSVM From wxjmfauth at gmail.com Thu Feb 26 01:46:58 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 25 Feb 2015 22:46:58 -0800 (PST) Subject: Python Worst Practices In-Reply-To: References: Message-ID: <646615ef-d3cf-427a-94e7-e8b353c9bb53@googlegroups.com> Le mercredi 25 f?vrier 2015 21:46:42 UTC+1, Mark Lawrence a ?crit?: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. I searched for a non ascii char. I did not find one [0, 61] slides. jmf From wxjmfauth at gmail.com Thu Feb 26 02:03:49 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Wed, 25 Feb 2015 23:03:49 -0800 (PST) Subject: Python Worst Practices In-Reply-To: References: Message-ID: <58c3b874-13f0-4a68-8b09-542c52714160@googlegroups.com> Le mercredi 25 f?vrier 2015 22:29:06 UTC+1, MRAB a ?crit?: > On 2015-02-25 20:45, Mark Lawrence wrote: > > http://www.slideshare.net/pydanny/python-worst-practices > > > > Any that should be added to this list? Any that be removed as not that bad? > > > We don't have numeric ZIP codes in the UK, but the entire world has > numeric telephone numbers, so that might be a better example of numbers > that aren't really numbers. > > Numeric dates can be ambiguous: dd/mm/yyyy or mm/dd/yyyy? The ISO > standard is clearer: yyyy-mm-dd. > > Handling text: "Unicode sandwich". > > UTF-8 is better than legacy encodings. I'm on the Unicode side. Unfortunately and objectively, I can not share this opinion. Most of people on this planet are not narrow minded ascii user. And what to say about the disastrous Unicode implementation of Unicode in Python? See https://groups.google.com/forum/#!topic/comp.lang.python/Avq1gRc2dzE jmf From storchaka at gmail.com Thu Feb 26 02:33:07 2015 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 26 Feb 2015 09:33:07 +0200 Subject: Bug in timsort!? In-Reply-To: References: <1cf84559-3a63-4799-a879-ae8e513d387e@googlegroups.com> Message-ID: On 25.02.15 18:56, Chris Kaynor wrote: > While the CPython implementation was only broken for arrays of length > larger than 2**49, aka, practically not broken, the Java > implementation (such as used on Android phones) was broken with arrays > of length > 67,108,864 at the time the post was made. While still very > large, an array of 67 million elements is well within the realm of > possibility today. The Java fixes (so far) have only extended the > number out, rather than actually fix the underlying problem - the > CPython fixes that were committed implement the full proven fix, so > CPython should now be able to handle infinite length arrays (once we > can build computers with that much storage...). No, it is limited to arrays with length < about 2**64. From lk.palao at gmail.com Thu Feb 26 02:47:12 2015 From: lk.palao at gmail.com (Leo Kris Palao) Date: Thu, 26 Feb 2015 15:47:12 +0800 Subject: GDAL Installation in Enthought Python Distribution Message-ID: Hi Python Users, Would like to request how to install GDAL in my Enthought Python Distribution (64-bit). I am having some problems making GDAL work. Or can you point me into a blog that describes how to set up GDAL in Enthought Python Distribution. Thanks for any help. -Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From amito2in at gmail.com Thu Feb 26 02:49:06 2015 From: amito2in at gmail.com (amito2in at gmail.com) Date: Wed, 25 Feb 2015 23:49:06 -0800 (PST) Subject: mod_python.publisher issue on ARM 64 bit machine Message-ID: <3864cde5-400e-4853-be07-6041f3cc9c31@googlegroups.com> Hi, We are using the following configurations on ARM 64 bit machine Apache Version:- 2.4.10 Python Version:- 2.7.4 Mod_Python:- 3.5.0 And we are observing the following error in mod_python.publisher while running our application Mod_python error: "PythonHandler mod_python.publisher" Traceback (most recent call last): File "/usr/bin/lib/python2.7/site-packages/mod_python/apache.py", line 421, in HandlerDispatch result = obj(req) File "/usr/bin/lib/python2.7/site-packages/mod_python/publisher.py", line 128, in handler path,module_name = os.path.split(req.filename) File "/usr/bin/lib/python2.7/posixpath.py", line 92, in split i = p.rfind('/') + 1 AttributeError: 'NoneType' object has no attribute 'rfind' However, mod_python.testhandler is working fine and its output is showing mod_python.publisher as available. mod_python.publisher available Yes mod_python.psp available Yes I came across the following link wherein it has suggested to make "_server" and "_interpreter" as public. I tried but it didn't work. http://osdir.com/ml/apache.mod-python.devel/2006-03/msg00250.html/ Do we have to use any particular version of Apache or Python on ARM 64 bit server or are we missing on any step in our current set-up? Please suggest Note:- We also tried with another version of mod_python(i.e. 3.4.1) From shiladittyasagar at gmail.com Thu Feb 26 02:49:21 2015 From: shiladittyasagar at gmail.com (Shiladittya Chakraborty) Date: Wed, 25 Feb 2015 23:49:21 -0800 (PST) Subject: Python STEP file color and module issue. Message-ID: <1759e3f0-c7b9-4008-92f9-707e083909af@googlegroups.com> In the latest version of pythonOCC-0.16.0-win32-py34 its supporting WEBGGL that's a great one but its showing only one color and also its does not have all the module when I am going to compare this pythonOCC-0.16.0-win32-py34 with pythonOCC-0.4.win32-py2.6. So my questions are - 1) How to get the remaining module? 2) Is it possible to STEP file read with exact color? -- Right now step file reading is possible with only one color. -- Is it possible to showing exact color using STEPCAFControl this module? From jvarghese at quixey.com Thu Feb 26 02:59:12 2015 From: jvarghese at quixey.com (jvarghese at quixey.com) Date: Wed, 25 Feb 2015 23:59:12 -0800 (PST) Subject: splunk_handler and logbook Message-ID: Hi, I am tring to use splunk(splunk_handler), my question is "Is there any way to integrate logbook with splunk_handler". The examples of splunk_handler uses python's logging module. Thanks From breamoreboy at yahoo.co.uk Thu Feb 26 03:54:35 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 26 Feb 2015 08:54:35 +0000 Subject: GDAL Installation in Enthought Python Distribution In-Reply-To: References: Message-ID: On 26/02/2015 07:47, Leo Kris Palao wrote: > Hi Python Users, > > Would like to request how to install GDAL in my Enthought Python > Distribution (64-bit). I am having some problems making GDAL work. Or > can you point me into a blog that describes how to set up GDAL in > Enthought Python Distribution. > > Thanks for any help. > -Leo > > Was it really neccessary to start a new thread one day after asking this question in a slightly different formt? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From wxjmfauth at gmail.com Thu Feb 26 03:58:23 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 26 Feb 2015 00:58:23 -0800 (PST) Subject: Toying with pypy Message-ID: I just try to reproduced the following with pypy. Unfortunately, that's impossible. D:\junk>d:\jm\wuni\jmtest2 Py 3.2.5 (default, May 15 2013, 23:06:03) [MSC v.1500 32 bit (Intel)] Quelques caract?res: ?abc??????????? Loop: empty string => quit ?>abc???z Votre entr?e ?tait : abc???z 7 caract?re(s) ?>abc\u03b1\u03b2\u03b3\u0430\u0431\u0432end Votre entr?e ?tait : abc??????end 12 caract?re(s) ?>z\\z\u0153z Votre entr?e ?tait : z\z?z 5 caract?re(s) ?>a\\009z Votre entr?e ?tait : a\009z 6 caract?re(s) ?>a\009z Wahrscheinlich falsches \uxxxx, (single, invalid backslash) ?>c:\\temp\\new.txt Votre entr?e ?tait : c:\temp\new.txt 15 caract?re(s) ?>etc Votre entr?e ?tait : etc 3 caract?re(s) ?> Fin D:\junk> From breamoreboy at yahoo.co.uk Thu Feb 26 03:59:14 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 26 Feb 2015 08:59:14 +0000 Subject: Python Worst Practices In-Reply-To: <54ee8b96$0$11124$c3e8da3@news.astraweb.com> References: <54ee8b96$0$11124$c3e8da3@news.astraweb.com> Message-ID: On 26/02/2015 02:57, Steven D'Aprano wrote: > Mark Lawrence wrote: > >> On 25/02/2015 20:45, Mark Lawrence wrote: >>> http://www.slideshare.net/pydanny/python-worst-practices >>> >>> Any that should be added to this list? Any that be removed as not that >>> bad? >>> >> >> Throwing in my own, how about built-in functions should not use "object" >> as the one and only argument, and a keyword argument at that. > > > Which built-in function is that? > memoryview see http://bugs.python.org/issue20408 -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Thu Feb 26 04:00:25 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 26 Feb 2015 09:00:25 +0000 Subject: Python Worst Practices In-Reply-To: <54EE8D63.2050208@davea.name> References: <54EE8D63.2050208@davea.name> Message-ID: On 26/02/2015 03:05, Dave Angel wrote: > On 02/25/2015 08:44 PM, Mark Lawrence wrote: >> On 25/02/2015 20:45, Mark Lawrence wrote: >>> http://www.slideshare.net/pydanny/python-worst-practices >>> >>> Any that should be added to this list? Any that be removed as not that >>> bad? >>> >> >> Throwing in my own, how about built-in functions should not use "object" >> as the one and only argument, and a keyword argument at that. >> > > def marry(object = False)... > > > if anybody has any cause to object, let him speak now... > Fell off me chair larfing. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From cl at isbd.net Thu Feb 26 05:09:27 2015 From: cl at isbd.net (cl at isbd.net) Date: Thu, 26 Feb 2015 10:09:27 +0000 Subject: Python Worst Practices References: Message-ID: Ian Kelly wrote: > On Wed, Feb 25, 2015 at 1:45 PM, Mark Lawrence wrote: > > http://www.slideshare.net/pydanny/python-worst-practices > > > > Any that should be added to this list? Any that be removed as not that bad? > > Using XML for configuration is a good example of a worst practice, but > using Python instead isn't best practice. There are good arguments > that a configuration language shouldn't be Turing-complete. See for > instance this blog post: http://taint.org/2011/02/18/001527a.html > I agree wholeheartedly about XML, it's just not designed for what half the world seems to be using it for. Rather like HTML in a way, that should have been a proper mark-up language. -- Chris Green ? From cl at isbd.net Thu Feb 26 05:12:16 2015 From: cl at isbd.net (cl at isbd.net) Date: Thu, 26 Feb 2015 10:12:16 +0000 Subject: Python Worst Practices References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> Message-ID: <0583sb-j98.ln1@esprimo.zbmc.eu> Ben Finney wrote: > Chris Angelico writes: > > > I'd really like to see a lot more presentations done in pure text. > > Maybe so. My request at the moment, though, is not for people to change > what's on their slides; rather, if they want people to retrieve them, > the slides should be downloadable easily (i.e. without a web app, > without a registration to some specific site). > ... and having downloaded them what do you view them with if they're not plain text? -- Chris Green ? From cfkaran2 at gmail.com Thu Feb 26 05:58:36 2015 From: cfkaran2 at gmail.com (Cem Karan) Date: Thu, 26 Feb 2015 05:58:36 -0500 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On Feb 26, 2015, at 12:36 AM, Gregory Ewing wrote: > Cem Karan wrote: >> I think I see what you're talking about now. Does WeakMethod >> (https://docs.python.org/3/library/weakref.html#weakref.WeakMethod) solve >> this problem? > > Yes, that looks like it would work. Cool! Thanks, Cem Karan From clawsicus at gmail.com Thu Feb 26 06:34:30 2015 From: clawsicus at gmail.com (Chris Laws) Date: Thu, 26 Feb 2015 22:04:30 +1030 Subject: asyncio POLLHUP question Message-ID: I have a system scenario where thousands of applications are running and via a service discovery mechanism they all get notified that a service they are all interesting in has come online. They all attempt to connect a TCP socket to the service. This happen virtually instantly. The problem that I see is that many of the applications that try to connect to the server get themselves into a state where they are consuming a lot of CPU. I am using Python 3.4.2, asyncio and have set the server backlog set to 4000 in an effort to accomodate the connection request backlog. I am actually using an event loop from aiozmq (but no ZMQ sockets in this scenaio) but under the covers this is just using epoll so it should really be the same as using the DefaultSelector. Using strace on the apps exhibiting issues I see that a socket is continuously triggering a POLLERR|POLLHUP event. This is the cause of the large CPU usage. The socket is the one that was attempting to connect to the new service that was just brought up. I am guessing that the POLLHUP is caused by the server having issues processing the volume of connect requests. I think I need to drop/close the socket causing the POLLHUP. However, from looking through the asyncio source code I don't see how I can do that from within the _selector.select() or _process_events() functions with only the knowledge of which fd is causing the issue. How do poll errors propagate up from the select loop? I can potentially unregister the fd but I don't think this will trigger the transport/protocol getting closed (as far as I can tell) which prevents my normal error handling scenarios from attempting to reconnect to the service. The asyncio select functions seem to ignore events other than EVENT_READ and EVENT_WRITE. Any help would be appreciated. Regards, Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Thu Feb 26 07:26:15 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 26 Feb 2015 23:26:15 +1100 Subject: Python Worst Practices References: <54ee60af$0$13001$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ef10e8$0$12984$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > On Thu, Feb 26, 2015 at 10:54 AM, Steven D'Aprano > wrote: >> - Violating the Rule of Demeter: don't talk to the dog's leg, talk to >> the dog. Or another way to put it: don't let the paper boy reach >> into your pocket for money. > > I'd call that code smell, rather than an automatic worst practice. Well, I did end my post with: And very possibly the worst practice of all: - Failing to understand when it is, and isn't, appropriate to break the rules and do what would otherwise be a bad practice. :-) > Suppose this: > > class Shelf: > def __init__(self): > self.items = [] # Empty shelf > > bookshelf = Shelf() > bookshelf.items.append(book) > > To enforce Demeter, you'd have to add a bunch of methods to the Shelf > whose sole purpose is to pass word along to the items list. Sure, it > makes some design sense to say "Add this book to the shelf" rather > than "Add this book to the items on the shelf", but all those lines of > code are potential bugs, and if you have to reimplement huge slabs of > functionality, that too is code smell. So there are times when it's > correct to reach into another object. Yes, well this comes down to the question of encapsulation and information-hiding. The advantage of exposing the list of items to the public is that anyone can add or remove items, sort them, reverse them, etc. The disadvantage is that you are now committed to keeping that list as part of the public API and you can't easily change the implementation. In this specific example, I'd probably keep the list as part of the Shelf API, although I'd be tempted to make self.items a read-only property. That will allow you to call list mutator methods, but prevent you from doing something silly like: bookshelf.items = 23 > But the times to use two dots are much rarer than the times to use one > dot (the paper boy shouldn't reach into your pocket for money, but > ThinkGeek has your credit card number on file so you can order more > conveniently), and I can't think of any example off-hand where you > would want more than three dots. The Law of Demeter is not really about counting dots. Ruby encourages chaining methods. Python doesn't, since built-ins typically don't return self. But in your own classes, you can have methods return self so you can chain them like this: mylist.append(spam).insert(1, eggs).append(cheese).sort().index(ham) Five dots or not, this is not a violation of Demeter. Likewise for long package names: from mylibrary.audiovisual.image.jpeg import Handler The Law of Demeter is more about information hiding. Clearly you don't hide *public* attributes of your class, otherwise they aren't public, so it's perfectly acceptable to say: myshelf.items.append(spam).insert(1, eggs).append(cheese).sort().index(ham) if items is public. But if the Shelf designer decides that the user shouldn't know anything about how the shelf stores its items, then the *first* dot violates the Law of Demeter. -- Steven From rustompmody at gmail.com Thu Feb 26 07:40:09 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 26 Feb 2015 04:40:09 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name><201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: On Wednesday, February 25, 2015 at 2:12:09 AM UTC+5:30, Dave Angel wrote: > On 02/24/2015 02:57 PM, Laura Creighton wrote: > > Dave Angel > > are you another Native English speaker living in a world where ASCII > > is enough? > > I'm a native English speaker, and 7 bits is not nearly enough. Even if > I didn't currently care, I have some history: > > No. CDC display code is enough. Who needs lowercase? > > No. Baudot code is enough. > > No, EBCDIC is good enough. Who cares about other companies. > > No, the "golf-ball" only holds this many characters. If we need more, > we can just get the operator to switch balls in the middle of printing. > > No. 2 digit years is enough. This world won't last till the millennium > anyway. > > No. 2k is all the EPROM you can have. Your code HAS to fit in it, and > only 1.5k RAM. > > No. 640k is more than anyone could need. > > No, you cannot use a punch card made on a model 26 keypunch in the same > deck as one made on a model 29. Too bad, many of the codes are > different. (This one cost me travel back and forth between two > different locations with different model keypunches) > > No. 8 bits is as much as we could ever use for characters. Who could > possibly need names or locations outside of this region? Or from > multiple places within it? > > 35 years ago I helped design a serial terminal that "spoke" Chinese, > using a two-byte encoding. But a single worldwide standard didn't come > until much later, and I cheered Unicode when it was finally unveiled. > > I've worked with many printers that could only print 70 or 80 unique > characters. The laser printer, and even the matrix printer are > relatively recent inventions. Wrote something up on why we should stop using ASCII: http://blog.languager.org/2015/02/universal-unicode.html (Yeah the world is a bit larger than a small bunch of islands off a half-continent. But this is not that discussion!) From mvoicem at gmail.com Thu Feb 26 08:12:24 2015 From: mvoicem at gmail.com (m) Date: Thu, 26 Feb 2015 14:12:24 +0100 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <54ef1bb8$0$2175$65785112@news.neostrada.pl> W dniu 25.02.2015 21:45, Mark Lawrence pisze: > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? > I disagree with slide 16. If I wanted to use long variable names, I would still code in Java. regards m. From rosuav at gmail.com Thu Feb 26 08:12:57 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 00:12:57 +1100 Subject: Python Worst Practices In-Reply-To: <54ef10e8$0$12984$c3e8da3$5496439d@news.astraweb.com> References: <54ee60af$0$13001$c3e8da3$5496439d@news.astraweb.com> <54ef10e8$0$12984$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 26, 2015 at 11:26 PM, Steven D'Aprano wrote: > Chris Angelico wrote: > >> But the times to use two dots are much rarer than the times to use one >> dot (the paper boy shouldn't reach into your pocket for money, but >> ThinkGeek has your credit card number on file so you can order more >> conveniently), and I can't think of any example off-hand where you >> would want more than three dots. > > The Law of Demeter is not really about counting dots. Ruby encourages > chaining methods. Python doesn't, since built-ins typically don't return > self. But in your own classes, you can have methods return self so you can > chain them like this: > > mylist.append(spam).insert(1, eggs).append(cheese).sort().index(ham) > > Five dots or not, this is not a violation of Demeter. Likewise for long > package names: > > from mylibrary.audiovisual.image.jpeg import Handler Yes, there are other places where you have lots of dots... I'm talking about the "rule of thumb" shorthand for describing the Law of Demeter, which is that you shouldn't have more than one dot before your method call. The chaining isn't that, because each one is a separate entity; but if you say "fred.house.bookshelf.items.append(book)", you're reaching in far too deep - you should be giving Fred the book to place on his own shelf. That's the only way where "counting dots" is a valid shorthand. It's mentioned in the Wikipedia article for the law: https://en.wikipedia.org/wiki/Law_of_Demeter#In_object-oriented_programming Can you offer a less ambiguous way to describe Demeter violations? By the "counting dots" style, Demeter demands one, I would be happy with two, and three or more strongly suggests a flawed API or overly-tight coupling - with the exception that module references don't get counted ("import os; os.path.append(p)" is one dot, because there's no point having an "os.add_path()" method). In some languages, those module references would be notated differently (os::path.append(p)), so simply counting dots would be closer to accurate. Is there a better Python description? ChrisA From rustompmody at gmail.com Thu Feb 26 08:15:35 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 26 Feb 2015 05:15:35 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se><54ECB134.5090304@davea.name><201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <7294b798-17a4-4743-901b-e189d37033e5@googlegroups.com> On Thursday, February 26, 2015 at 6:10:25 PM UTC+5:30, Rustom Mody wrote: > On Wednesday, February 25, 2015 at 2:12:09 AM UTC+5:30, Dave Angel wrote: > > On 02/24/2015 02:57 PM, Laura Creighton wrote: > > > Dave Angel > > > are you another Native English speaker living in a world where ASCII > > > is enough? > > > > I'm a native English speaker, and 7 bits is not nearly enough. Even if > > I didn't currently care, I have some history: > > > > No. CDC display code is enough. Who needs lowercase? > > > > No. Baudot code is enough. > > > > No, EBCDIC is good enough. Who cares about other companies. > > > > No, the "golf-ball" only holds this many characters. If we need more, > > we can just get the operator to switch balls in the middle of printing. > > > > No. 2 digit years is enough. This world won't last till the millennium > > anyway. > > > > No. 2k is all the EPROM you can have. Your code HAS to fit in it, and > > only 1.5k RAM. > > > > No. 640k is more than anyone could need. > > > > No, you cannot use a punch card made on a model 26 keypunch in the same > > deck as one made on a model 29. Too bad, many of the codes are > > different. (This one cost me travel back and forth between two > > different locations with different model keypunches) > > > > No. 8 bits is as much as we could ever use for characters. Who could > > possibly need names or locations outside of this region? Or from > > multiple places within it? > > > > 35 years ago I helped design a serial terminal that "spoke" Chinese, > > using a two-byte encoding. But a single worldwide standard didn't come > > until much later, and I cheered Unicode when it was finally unveiled. > > > > I've worked with many printers that could only print 70 or 80 unique > > characters. The laser printer, and even the matrix printer are > > relatively recent inventions. > > Wrote something up on why we should stop using ASCII: > http://blog.languager.org/2015/02/universal-unicode.html Dave's list above of instances of 'poverty is a good idea' turning out stupid and narrow-minded in hindsight is neat. Thought I'd ack that explicitly. From rosuav at gmail.com Thu Feb 26 08:24:03 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 00:24:03 +1100 Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: On Thu, Feb 26, 2015 at 11:40 PM, Rustom Mody wrote: > Wrote something up on why we should stop using ASCII: > http://blog.languager.org/2015/02/universal-unicode.html >From that post: """ 5.1 Gibberish When going from the original 2-byte unicode (around version 3?) to the one having supplemental planes, the unicode consortium added blocks such as * Egyptian hieroglyphs * Cuneiform * Shavian * Deseret * Mahjong * Klingon To me (a layman) it looks unprofessional ? as though they are playing games ? that billions of computing devices, each having billions of storage words should have their storage wasted on blocks such as these. """ The shift from Unicode as a 16-bit code to having multiple planes came in with Unicode 2.0, but the various blocks were assigned separately: * Egyptian hieroglyphs: Unicode 5.2 * Cuneiform: Unicode 5.0 * Shavian: Unicode 4.0 * Deseret: Unicode 3.1 * Mahjong Tiles: Unicode 5.1 * Klingon: Not part of any current standard However, I don't think historians will appreciate you calling all of these "gibberish". To adequately describe and discuss old texts without these Unicode blocks, we'd have to either do everything with images, or craft some kind of reversible transliteration system and have dedicated software to render the texts on screen. Instead, what we have is a well-known and standardized system for transliterating all of these into numbers (code points), and rendering them becomes a simple matter of installing an appropriate font. Also, how does assigning meanings to codepoints "waste storage"? As soon as Unicode 2.0 hit and 16-bit code units stopped being sufficient, everyone needed to allocate storage - either 32 bits per character, or some other system - and the fact that some codepoints were unassigned had absolutely no impact on that. This is decidedly NOT unprofessional, and it's not wasteful either. ChrisA From rosuav at gmail.com Thu Feb 26 08:37:39 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 00:37:39 +1100 Subject: Python Worst Practices In-Reply-To: <54ef1bb8$0$2175$65785112@news.neostrada.pl> References: <54ef1bb8$0$2175$65785112@news.neostrada.pl> Message-ID: On Fri, Feb 27, 2015 at 12:12 AM, m wrote: > W dniu 25.02.2015 21:45, Mark Lawrence pisze: >> >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that >> bad? >> > > > I disagree with slide 16. If I wanted to use long variable names, I would > still code in Java. Clearly you aren't bothered by ambiguities, given that your name is "m". You're lower-case m, and the James Bond character is upper-case M... yeah, this isn't going to be a problem, with seven billion people on the planet! In case it's not obvious from slide 17, the author is advocating neither the ridiculously short, nor the ridiculously long. This is a topic that you could go into great detail on, but a general rule of thumb is that short names go with short-lived variables, and longer names go with large-scope variables. [1] So your function names shouldn't be single letters, but your loop counters can and should be short: def discard_all_spam(): for msg in self.messages: if msg.is_spam(): ms.discard() And of course, the use of "i" as an integer loop index dates back so far and is so well known that you don't need anything else: def get_password(): for i in range(4): if i: print("%d wrong tries...") s = input("What's the password? ") if validate_password(s): return s print("Too many wrong tries, go away.") This isn't Java coding. ChrisA [1] Yes, Python doesn't have variables per se. But how else am I supposed to differentiate between the name and the concept of a name binding? From invalid at invalid.invalid Thu Feb 26 09:37:11 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Thu, 26 Feb 2015 14:37:11 +0000 (UTC) Subject: Python Worst Practices References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> Message-ID: On 2015-02-26, Ben Finney wrote: > Chris Angelico writes: > >> IMO the whole system of boolean logic in shell scripts is a massive >> pile of hacks. > > Agreed. It bears all the hallmarks of a system which has been > extended to become a complete programming language only with extreme > reluctance on its part. > > I continue to be impressed by how capable and powerful Unix shell is > as a full programming language. Especially it is sorely missed on > other OSes which lack a capable shell. > > But it could never be called ?elegant?. Unless you've spent all day working on PHP code. -- Grant Edwards grant.b.edwards Yow! All of life is a blur at of Republicans and meat! gmail.com From malik.a.rumi at gmail.com Thu Feb 26 09:57:44 2015 From: malik.a.rumi at gmail.com (Malik Rumi) Date: Thu, 26 Feb 2015 06:57:44 -0800 (PST) Subject: Is anyone else unable to log into the bug tracker? In-Reply-To: <54b0850a$0$13002$c3e8da3$5496439d@news.astraweb.com> References: <54b0850a$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Friday, January 9, 2015 at 7:49:09 PM UTC-6, Steven D'Aprano wrote: > I'm having trouble logging into the bug tracker. Is anyone else having the > same problem, that is, your user name and password worked earlier but > doesn't work now? > > http://bugs.python.org/ > > (Yes, I've checked the capslock key.) > > > > Before I request a new password, I want to check whether it is me or > everyone. > > > -- > Steven On Friday, January 9, 2015 at 7:49:09 PM UTC-6, Steven D'Aprano wrote: > I'm having trouble logging into the bug tracker. Is anyone else having the > same problem, that is, your user name and password worked earlier but > doesn't work now? > > http://bugs.python.org/ > > (Yes, I've checked the capslock key.) > > > > Before I request a new password, I want to check whether it is me or > everyone. > > > -- > Steven On Friday, January 9, 2015 at 7:49:09 PM UTC-6, Steven D'Aprano wrote: > I'm having trouble logging into the bug tracker. Is anyone else having the > same problem, that is, your user name and password worked earlier but > doesn't work now? > > http://bugs.python.org/ > > (Yes, I've checked the capslock key.) > > > > Before I request a new password, I want to check whether it is me or > everyone. > > > -- > Steven I am having this problem, even after I requested a new password. All I get is 'invalid login'. How did you resolve? Thx. From jason.swails at gmail.com Thu Feb 26 10:06:06 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 26 Feb 2015 10:06:06 -0500 Subject: Parallelization of Python on GPU? In-Reply-To: <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> Message-ID: <1424963166.30927.73.camel@gmail.com> On Thu, 2015-02-26 at 14:02 +1100, Steven D'Aprano wrote: > John Ladasky wrote: > > > > What I would REALLY like to do is to take advantage of my GPU. > > I can't help you with that, but I would like to point out that GPUs > typically don't support IEE-754 maths, which means that while they are > likely significantly faster, they're also likely significantly less > accurate. Any any two different brands/models of GPU are likely to give > different results. (Possibly not *very* different, but considering the mess > that floating point maths was prior to IEEE-754, possibly *very* different.) This hasn't been true in NVidia GPUs manufactured since ca. 2008. > Personally, I wouldn't trust GPU floating point for serious work. Maybe for > quick and dirty exploration of the data, but I'd then want to repeat any > calculations using the main CPU before using the numbers anywhere :-) There is a *huge* dash toward GPU computing in the scientific computing sector. Since I started as a graduate student in computational chemistry/physics in 2008, I watched as state-of-the-art supercomputers running tens of thousands to hundreds of thousands of cores were overtaken in performance by a $500 GPU (today the GTX 780 or 980) you can put in a desktop. I went from running all of my calculations on a CPU cluster in 2009 to running 90% of my calculations on a GPU by the time I graduated in 2013... and for people without as ready access to supercomputers as myself the move was even more pronounced. This work is very serious, and numerical precision is typically of immense importance. See, e.g., http://www.sciencedirect.com/science/article/pii/S0010465512003098 and http://pubs.acs.org/doi/abs/10.1021/ct400314y In our software, we can run simulations on a GPU or a CPU and the results are *literally* indistinguishable. The transition to GPUs was accompanied by a series of studies that investigated precisely your concerns... we would never have started using GPUs if we didn't trust GPU numbers as much as we did from the CPU. And NVidia is embracing this revolution (obviously) -- they are putting a lot of time, effort, and money into ensuring the success of GPU high performance computing. It is here to stay in the immediate future, and refusing to use the technology will leave those that *could* benefit from it at a severe disadvantage. (That said, GPUs aren't good at everything, and CPUs are also here to stay.) And GPU performance gains are outpacing CPU performance gains -- I've seen about two orders of magnitude improvement in computational throughput over the past 6 years through the introduction of GPU computing and improvements in GPU hardware. All the best, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher From malik.a.rumi at gmail.com Thu Feb 26 10:10:33 2015 From: malik.a.rumi at gmail.com (Malik Rumi) Date: Thu, 26 Feb 2015 07:10:33 -0800 (PST) Subject: Windows permission error, 64 bit, psycopg2, python 3.4.2 Message-ID: <6a27dcaa-4652-4dc4-8c74-fd0a68ebb51a@googlegroups.com> I am one of those struggling with compile issues with python on 64 bit windows. I have not been able to get the solutions mentioned on Stack Overflow to work because installing Windows SDK 7.1 fails for me. So I stumbled across a precompiled psycopg2, and that reported that it worked, but then I got two permission errors. Then I read that this was a bug in python (issue 14252) that had been fixed, but I don't think this is the same error. That one specifically refers to subprocess.py and I don't have that in my traceback. I have v3.4.2. On top of everything else, despite requesting a new password, all I get from the big tracker is 'invalid login'. In any event, running "import psycopg2" returns 'import error, no module named psycopg2'. Microsoft Windows [Version 6.3.9600] (c) 2013 Microsoft Corporation. All rights reserved. C:\Users\Semantic>pip install git+https://github.com/nwcell/psycopg2-windows.git @win64-py34#egg=psycopg2 Downloading/unpacking psycopg2 from git+https://github.com/nwcell/psycopg2-windo ws.git at win64-py34 Cloning https://github.com/nwcell/psycopg2-windows.git (to win64-py34) to c:\u sers\semantic\appdata\local\temp\pip_build_semantic\psycopg2 Running setup.py (path:C:\Users\Semantic\AppData\Local\Temp\pip_build_Semantic \psycopg2\setup.py) egg_info for package psycopg2 C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt ion: 'summary' warnings.warn(msg) Installing collected packages: psycopg2 Running setup.py install for psycopg2 C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt ion: 'summary' warnings.warn(msg) Successfully installed psycopg2 Cleaning up... Exception: Traceback (most recent call last): File "C:\Python34\lib\shutil.py", line 370, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Semantic\\AppData\\L ocal\\Temp\\pip_build_Semantic\\psycopg2\\.git\\objects\\pack\\pack-be4d3da4a06b 4c9ec4c06040dbf6685eeccca068.idx' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Python34\lib\site-packages\pip\basecommand.py", line 122, in main status = self.run(options, args) File "C:\Python34\lib\site-packages\pip\commands\install.py", line 302, in run requirement_set.cleanup_files(bundle=self.bundle) File "C:\Python34\lib\site-packages\pip\req.py", line 1333, in cleanup_files rmtree(dir) File "C:\Python34\lib\site-packages\pip\util.py", line 43, in rmtree onerror=rmtree_errorhandler) File "C:\Python34\lib\shutil.py", line 477, in rmtree return _rmtree_unsafe(path, onerror) File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Python34\lib\shutil.py", line 372, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Python34\lib\site-packages\pip\util.py", line 53, in rmtree_errorhand ler (exctype is PermissionError and value.args[3] == 5) #python3.3 IndexError: tuple index out of range From mal at europython.eu Thu Feb 26 10:16:22 2015 From: mal at europython.eu (M.-A. Lemburg) Date: Thu, 26 Feb 2015 16:16:22 +0100 Subject: EuroPython 2015: Launch preparations are underway Message-ID: <54EF38C6.6020808@europython.eu> The EuroPython Workgroups are busy preparing the launch of the website. Just launched in mid-January, all workgroups (WGs) are fully under steam by now, working hard to make EuroPython 2015 a fabulous event. http://ep2015.europython.eu/ Community building the conference --------------------------------- The *On-site Team WG* is doing a wonderful job getting us the best possible deals in Bilbao, the *Web WG* is knee deep into code and docker containers setting up the website, the *Marketing & Design WG* working with the designers to create wonderful logos and brochures, the *Program WG* contacting keynote speakers and creating the call for proposals, the *Finance WG* building the budget and making sure the conference stays affordable for everyone, the *Support WG* setting up the online help desk to answer your questions, the *Communications WG* preparing to create a constant stream of exciting news updates, the *Administration WG* is managing the many accounts, contracts and services needed to run the organization. The *Financial Aid WG* and *Media WG* are preparing to start their part of the conference organization later in March. http://www.europython-society.org/workgroups The WGs are all staffed with members from the ACPySS on-site team, the EuroPython Society and volunteers from the EuroPython community to drive the organization forward and we?re getting a lot done in a very short time frame. More help needed ---------------- We are very happy with the help we are getting from the community, but there still is a lot more to be done. If you want to help us build a great EuroPython conference, please consider joining one of the above workgroups: http://www.europython-society.org/workgroups Stay tuned and be sure to follow the EuroPython Blog for updates on the conference: http://blog.europython.eu/ Enjoy, - EuroPython Society (EPS) http://www.europython-society.org/ From larry.martell at gmail.com Thu Feb 26 10:23:35 2015 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 26 Feb 2015 10:23:35 -0500 Subject: Installing PIL without internet access Message-ID: I have a host that has no access to the internet and I need to install PIL on it. I have an identical host that is on the internet and I have installed it there (with pip). Is there a way I can copy files from the connected host to a flash drive and then copy them to the unconnected host and have PIL working there? Which files would I copy for that? This is on CentOS 6.5, python 2.7 Thanks! From rosuav at gmail.com Thu Feb 26 10:27:02 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 02:27:02 +1100 Subject: EuroPython 2015: Launch preparations are underway In-Reply-To: <54EF38C6.6020808@europython.eu> References: <54EF38C6.6020808@europython.eu> Message-ID: On Fri, Feb 27, 2015 at 2:16 AM, M.-A. Lemburg wrote: > [ a whole lot of relatively sane text ] Sadly, this was not what I wanted to see, based on the subject line. I wanted to know about the snake you guys were about to send into space! ChrisA From jason.swails at gmail.com Thu Feb 26 10:27:14 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 26 Feb 2015 10:27:14 -0500 Subject: Parallelization of Python on GPU? In-Reply-To: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: <1424964434.30927.89.camel@gmail.com> On Wed, 2015-02-25 at 18:35 -0800, John Ladasky wrote: > I've been working with machine learning for a while. Many of the > standard packages (e.g., scikit-learn) have fitting algorithms which > run in single threads. These algorithms are not themselves > parallelized. Perhaps, due to their unique mathematical requirements, > they cannot be paralleized. > > When one is investigating several potential models of one's data with > various settings for free parameters, it is still sometimes possible > to speed things up. On a modern machine, one can use Python's > multiprocessing.Pool to run separate instances of scikit-learn fits. > I am currently using ten of the twelve 3.3 GHz CPU cores on my machine > to do just that. And I can still browse the web with no observable > lag. :^) > > Still, I'm waiting hours for jobs to finish. Support vector > regression fitting is hard. > > What I would REALLY like to do is to take advantage of my GPU. My > NVidia graphics card has 1152 cores and a 1.0 GHz clock. I wouldn't > mind borrowing a few hundred of those GPU cores at a time, and see > what they can do. In theory, I calculate that I can speed up the job > by another five-fold. > > The trick is that each process would need to run some PYTHON code, not > CUDA or OpenCL. The child process code isn't particularly fancy. (I > should, for example, be able to switch that portion of my code to > static typing.) > > What is the most effective way to accomplish this task? GPU computing is a lot more than simply saying "run this on a GPU". To realize the performance gains promised by a GPU, you need to tailor your algorithms to take advantage of their hardware... SIMD reigns supreme where thread divergence and branching are far more expensive than they are in CPU computing. So even if you decide to somehow translate your Python code into a CUDA kernel, there is a good chance that you will be woefully disappointed in the resulting speedup (or even moreso if you actually get a slowdown :)). For example, a simple reduction is more expensive on a GPU than it is on a CPU for small arrays. A dot product, for example, has a part that's super fast on the GPU (element-by-element multiplication), and then a part that gets a lot slower (summing up all elements of the resulting multiplication). Each core on the GPU is a lot slower than a CPU (which is why a 1000-CUDA-core GPU doesn't run anywhere near 1000x faster than a CPU), so you really only get gains when they can all work efficiently together. Another example -- matrix multiplies are *fast*. Diagonalizations are slow (which is why in my field where diagonalizations are common requirements, they are often done on the CPU while *building* the matrix is done on the GPU). > > I came across a reference to a package called "Urutu" which may be > what I need, however it doesn't look like it is widely supported. Urutu seems to be built on PyCUDA and PyOpenCL (which are both written by the same person; Andreas Kloeckner at UIUC in the United States). Another package I would suggest looking into is numba, from Continuum Analytics: https://github.com/numba/numba. Unlike Urutu, their package is built on LLVM and Python bindings they've written to implement numpy-aware JIT capabilities. I believe they also permit compiling down to a GPU kernel through LLVM. One downside I've experienced with that package is that LLVM does not yet have a stable API (as I understand it), so they often lag behind support for the latest versions of LLVM. > > I would love it if the Python developers themselves added the ability > to spawn GPU processes to the Multiprocessing module! I would be stunned if this actually happened. If you're worried about performance, you get at least an order of magnitude performance boost by going to numpy or writing the kernel directly in C or Fortran. CPython itself just isn't structured to run on a GPU... maybe pypy will tackle that at some point in the probably-distant future. All the best, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher From af300wsm at gmail.com Thu Feb 26 10:57:45 2015 From: af300wsm at gmail.com (af300wsm at gmail.com) Date: Thu, 26 Feb 2015 07:57:45 -0800 (PST) Subject: Building C++ modules for python using GNU autotools, automake, whatever Message-ID: <364c8f3d-808e-4513-9041-6d8cdfdc42b7@googlegroups.com> Hi, I'm a complete neophyte to the whole use of GNU autotools/automake/auto... . (I'm not sure what it should be called anymore.) Regardless, I'm porting a library project, for which I'm a team member, to using this toolset for building in Linux. I'm to the point now of writing the Makefile.am file for the actual library. (There are several other static libraries compiled first that are sucked into this shared object file.) I found some references here: http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Python.html, which seemed to be just what I was after. However, I've got a big question about a file named "module.la" instead of "module.so" which is what we compile it to now. I guess I should have mentioned some background. Currently, we build this tool through some homegrown makefiles. This has worked, but distribution is difficult and our product must now run on an embedded platform (so building it cleanly requires the use of autotools). Basically, I need this thing to install to /usr/lib/python2.6/site-packages when the user invokes "make install". I thought the variables and primaries discussed at the link above were what I needed. However, what is a "*.la"? I'm reading up on libtool now, but will it function the same way as a *.so? I need pointers on where to go from here. Thanks, Andy From info at wingware.com Thu Feb 26 11:00:07 2015 From: info at wingware.com (Wingware) Date: Thu, 26 Feb 2015 11:00:07 -0500 Subject: ANN: Wing IDE 5.1.2 released Message-ID: <54EF4307.1000107@wingware.com> Hi, Wingware has released version 5.1.2 of Wing IDE, our cross-platform integrated development environment for the Python programming language. Wing IDE features a professional code editor with vi, emacs, visual studio, and other key bindings, auto-completion, call tips, context-sensitive auto-editing, goto-definition, find uses, refactoring, a powerful debugger, version control, unit testing, search, project management, and many other features. This minor release includes the following improvements: Support for recent Google App Engine versions Expanded and improved static analysis for PyQt Added class and instance attributes to the Find Symbol dialog Support recursive invocation of snippets, auto-invocation arg entry, and field-based auto-editing operations (e.g. :try applied to a selected range) Support for python3-pylint Code sign all exe, dll, and pyd files on Windows Fix a number of child process debugging scenarios Fix source assistant formatting of PEP287 fields with long fieldname Fix indent level for pasted text after single undo for indent adjustment Fix introduce variable refactoring and if (exp): statements About 12 other bug fixes; see http://wingware.com/pub/wingide/5.1.2/CHANGELOG.txt What's New in Wing 5.1: Wing IDE 5.1 adds multi-process and child process debugging, syntax highlighting in the shells, persistent time-stamped unit test results, auto-conversion of indents on paste, an XCode keyboard personality, support for Flask, Django 1.7 & recent Google App Engine versions, improved auto-completion for PyQt, recursive snippet invocation, and many other minor features and improvements. For details see http://wingware.com/news/2015-02-25 Free trial: http://wingware.com/wingide/trial Downloads: http://wingware.com/downloads Feature list: http://wingware.com/wingide/features Sales: http://wingware.com/store/purchase Upgrades: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From steve+comp.lang.python at pearwood.info Thu Feb 26 11:13:52 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 27 Feb 2015 03:13:52 +1100 Subject: Is anyone else unable to log into the bug tracker? References: <54b0850a$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54ef4641$0$13004$c3e8da3$5496439d@news.astraweb.com> Malik Rumi wrote: > On Friday, January 9, 2015 at 7:49:09 PM UTC-6, Steven D'Aprano wrote: >> I'm having trouble logging into the bug tracker. Is anyone else having >> the same problem, that is, your user name and password worked earlier but >> doesn't work now? >> >> http://bugs.python.org/ >> >> (Yes, I've checked the capslock key.) [...] > I am having this problem, even after I requested a new password. All I get > is 'invalid login'. How did you resolve? Thx. I was suffering from a PEBCAK error, and was using the wrong password. Once I started using the right one, it just worked for me. I seem to recall that you need to accept cookies for the bugtracker to log you in. Try that and see if it helps. Sorry that I can't be of more help. -- Steven From jason.swails at gmail.com Thu Feb 26 11:35:03 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 26 Feb 2015 11:35:03 -0500 Subject: Building C++ modules for python using GNU autotools, automake, whatever In-Reply-To: <364c8f3d-808e-4513-9041-6d8cdfdc42b7@googlegroups.com> References: <364c8f3d-808e-4513-9041-6d8cdfdc42b7@googlegroups.com> Message-ID: <1424968503.30927.114.camel@gmail.com> On Thu, 2015-02-26 at 07:57 -0800, af300wsm at gmail.com wrote: > Hi, > > I'm a complete neophyte to the whole use of GNU > autotools/automake/auto... . (I'm not sure what it should be called > anymore.) Regardless, I'm porting a library project, for which I'm a > team member, to using this toolset for building in Linux. I'm to the > point now of writing the Makefile.am file for the actual library. > (There are several other static libraries compiled first that are > sucked into this shared object file.) > > I found some references here: > http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Python.html, which seemed to be just what I was after. However, I've got a big question about a file named "module.la" instead of "module.so" which is what we compile it to now. I certainly hope module.la is not what it gets compiled to. Open it up with a text editor :). It's just basically a description of the library that libtool makes use of. In the projects that I build, the .la files are all associated with a .a archive or a .so (/.dylib for Macs). Obviously, static archives won't work for Python (and, in particular, I believe you need to compile all of the objects as position independent code, so you need to make sure the appropriate PIC flag is given to the compiler... for g++ that would be -fPIC). > > I guess I should have mentioned some background. Currently, we build > this tool through some homegrown makefiles. This has worked, but > distribution is difficult and our product must now run on an embedded > platform (so building it cleanly requires the use of autotools). > > Basically, I need this thing to install > to /usr/lib/python2.6/site-packages when the user invokes "make > install". I thought the variables and primaries discussed at the link > above were what I needed. However, what is a "*.la"? I'm reading up > on libtool now, but will it function the same way as a *.so? To libtool, yes... provided that you *also* have the .so with the same base name as the .la. I don't think compilers themselves make any use of .la files, though. HTH, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher From sturla.molden at gmail.com Thu Feb 26 11:40:29 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 26 Feb 2015 16:40:29 +0000 (UTC) Subject: Parallelization of Python on GPU? References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: <1346719069446661157.182796sturla.molden-gmail.com@news.gmane.org> If you are doing SVM regression with scikit-learn you are using libSVM. There is a CUDA accelerated version of this C library here: http://mklab.iti.gr/project/GPU-LIBSVM You can presumably reuse the wrapping code from scikit-learn. Sturla John Ladasky wrote: > I've been working with machine learning for a while. Many of the > standard packages (e.g., scikit-learn) have fitting algorithms which run > in single threads. These algorithms are not themselves parallelized. > Perhaps, due to their unique mathematical requirements, they cannot be paralleized. > > When one is investigating several potential models of one's data with > various settings for free parameters, it is still sometimes possible to > speed things up. On a modern machine, one can use Python's > multiprocessing.Pool to run separate instances of scikit-learn fits. I > am currently using ten of the twelve 3.3 GHz CPU cores on my machine to > do just that. And I can still browse the web with no observable lag. :^) > > Still, I'm waiting hours for jobs to finish. Support vector regression fitting is hard. > > What I would REALLY like to do is to take advantage of my GPU. My NVidia > graphics card has 1152 cores and a 1.0 GHz clock. I wouldn't mind > borrowing a few hundred of those GPU cores at a time, and see what they > can do. In theory, I calculate that I can speed up the job by another five-fold. > > The trick is that each process would need to run some PYTHON code, not > CUDA or OpenCL. The child process code isn't particularly fancy. (I > should, for example, be able to switch that portion of my code to static typing.) > > What is the most effective way to accomplish this task? > > I came across a reference to a package called "Urutu" which may be what I > need, however it doesn't look like it is widely supported. > > I would love it if the Python developers themselves added the ability to > spawn GPU processes to the Multiprocessing module! > > Thanks for any advice and comments. From sam.raker at gmail.com Thu Feb 26 11:45:56 2015 From: sam.raker at gmail.com (Sam Raker) Date: Thu, 26 Feb 2015 08:45:56 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <0c1f4147-2d5d-4fa6-afb9-2275d878e2c1@googlegroups.com> I'm 100% in favor of expanding Unicode until the sun goes dark. Doing so helps solve the problems affecting speakers of "underserved" languages--access and language preservation. Speakers of Mongolian, Cherokee, Georgian, etc. all deserve to be able to interact with technology in their native languages as much as we speakers of ASCII-friendly languages do. Unicode support also makes writing papers on, dictionaries of, and new texts in such languages much easier, which helps the fight against language extinction, which is a sadly pressing issue. Also, like, computers are big. Get an external drive for your high-resolution PDF collection of Medieval manuscripts if you feel like you're running out of space. A few extra codepoints aren't going to be the straw that breaks the camel's back. On Thursday, February 26, 2015 at 8:24:34 AM UTC-5, Chris Angelico wrote: > On Thu, Feb 26, 2015 at 11:40 PM, Rustom Mody wrote: > > Wrote something up on why we should stop using ASCII: > > http://blog.languager.org/2015/02/universal-unicode.html > > >From that post: > > """ > 5.1 Gibberish > > When going from the original 2-byte unicode (around version 3?) to the > one having supplemental planes, the unicode consortium added blocks > such as > > * Egyptian hieroglyphs > * Cuneiform > * Shavian > * Deseret > * Mahjong > * Klingon > > To me (a layman) it looks unprofessional - as though they are playing > games - that billions of computing devices, each having billions of > storage words should have their storage wasted on blocks such as > these. > """ > > The shift from Unicode as a 16-bit code to having multiple planes came > in with Unicode 2.0, but the various blocks were assigned separately: > * Egyptian hieroglyphs: Unicode 5.2 > * Cuneiform: Unicode 5.0 > * Shavian: Unicode 4.0 > * Deseret: Unicode 3.1 > * Mahjong Tiles: Unicode 5.1 > * Klingon: Not part of any current standard > > However, I don't think historians will appreciate you calling all of > these "gibberish". To adequately describe and discuss old texts > without these Unicode blocks, we'd have to either do everything with > images, or craft some kind of reversible transliteration system and > have dedicated software to render the texts on screen. Instead, what > we have is a well-known and standardized system for transliterating > all of these into numbers (code points), and rendering them becomes a > simple matter of installing an appropriate font. > > Also, how does assigning meanings to codepoints "waste storage"? As > soon as Unicode 2.0 hit and 16-bit code units stopped being > sufficient, everyone needed to allocate storage - either 32 bits per > character, or some other system - and the fact that some codepoints > were unassigned had absolutely no impact on that. This is decidedly > NOT unprofessional, and it's not wasteful either. > > ChrisA From tjreedy at udel.edu Thu Feb 26 11:47:04 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Feb 2015 11:47:04 -0500 Subject: GDAL Installation in Enthought Python Distribution In-Reply-To: References: Message-ID: On 2/26/2015 2:47 AM, Leo Kris Palao wrote: > Would like to request how to install GDAL in my Enthought Python > Distribution (64-bit). The best place to ask about the Enthought Python Distribution is a list devoted to the E. P. D. -- Terry Jan Reedy From skip.montanaro at gmail.com Thu Feb 26 11:48:59 2015 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 26 Feb 2015 10:48:59 -0600 Subject: Is anyone else unable to log into the bug tracker? In-Reply-To: References: <54b0850a$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: I have not had problems, but I use the Google login (Open ID, I presume) option. Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Thu Feb 26 11:53:27 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 26 Feb 2015 16:53:27 +0000 (UTC) Subject: Parallelization of Python on GPU? References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> Message-ID: <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> GPU computing is great if you have the following: 1. Your data structures are arrays floating point numbers. 2. You have a data-parallel problem. 3. You are happy with single precision. 4. You have time to code erything in CUDA or OpenCL. 5. You have enough video RAM to store your data. For Python the easiest solution is to use Numba Pro. Sturla Jason Swails wrote: > On Thu, 2015-02-26 at 14:02 +1100, Steven D'Aprano wrote: >> John Ladasky wrote: >> >> >>> What I would REALLY like to do is to take advantage of my GPU. >> >> I can't help you with that, but I would like to point out that GPUs >> typically don't support IEE-754 maths, which means that while they are >> likely significantly faster, they're also likely significantly less >> accurate. Any any two different brands/models of GPU are likely to give >> different results. (Possibly not *very* different, but considering the mess >> that floating point maths was prior to IEEE-754, possibly *very* different.) > > This hasn't been true in NVidia GPUs manufactured since ca. 2008. > >> Personally, I wouldn't trust GPU floating point for serious work. Maybe for >> quick and dirty exploration of the data, but I'd then want to repeat any >> calculations using the main CPU before using the numbers anywhere :-) > > There is a *huge* dash toward GPU computing in the scientific computing > sector. Since I started as a graduate student in computational > chemistry/physics in 2008, I watched as state-of-the-art supercomputers > running tens of thousands to hundreds of thousands of cores were > overtaken in performance by a $500 GPU (today the GTX 780 or 980) you > can put in a desktop. I went from running all of my calculations on a > CPU cluster in 2009 to running 90% of my calculations on a GPU by the > time I graduated in 2013... and for people without as ready access to > supercomputers as myself the move was even more pronounced. > > This work is very serious, and numerical precision is typically of > immense importance. See, e.g., > http://www.sciencedirect.com/science/article/pii/S0010465512003098 and > http://pubs.acs.org/doi/abs/10.1021/ct400314y > > In our software, we can run simulations on a GPU or a CPU and the > results are *literally* indistinguishable. The transition to GPUs was > accompanied by a series of studies that investigated precisely your > concerns... we would never have started using GPUs if we didn't trust > GPU numbers as much as we did from the CPU. > > And NVidia is embracing this revolution (obviously) -- they are putting > a lot of time, effort, and money into ensuring the success of GPU high > performance computing. It is here to stay in the immediate future, and > refusing to use the technology will leave those that *could* benefit > from it at a severe disadvantage. (That said, GPUs aren't good at > everything, and CPUs are also here to stay.) > > And GPU performance gains are outpacing CPU performance gains -- I've > seen about two orders of magnitude improvement in computational > throughput over the past 6 years through the introduction of GPU > computing and improvements in GPU hardware. > > All the best, > Jason From python at mrabarnett.plus.com Thu Feb 26 11:57:29 2015 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 26 Feb 2015 16:57:29 +0000 Subject: Installing PIL without internet access In-Reply-To: References: Message-ID: <54EF5079.7080402@mrabarnett.plus.com> On 2015-02-26 15:23, Larry Martell wrote: > I have a host that has no access to the internet and I need to install > PIL on it. I have an identical host that is on the internet and I have > installed it there (with pip). Is there a way I can copy files from > the connected host to a flash drive and then copy them to the > unconnected host and have PIL working there? Which files would I copy > for that? > > This is on CentOS 6.5, python 2.7 > Have a look here: https://pip.pypa.io/en/latest/reference/pip_install.html#pip-install-options It says that you can install from a downloaded file, e.g.: pip install ./downloads/SomePackage-1.0.4.tar.gz From tjreedy at udel.edu Thu Feb 26 12:02:25 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Feb 2015 12:02:25 -0500 Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: On 2/26/2015 8:24 AM, Chris Angelico wrote: > On Thu, Feb 26, 2015 at 11:40 PM, Rustom Mody wrote: >> Wrote something up on why we should stop using ASCII: >> http://blog.languager.org/2015/02/universal-unicode.html I think that the main point of the post, that many Unicode chars are truly planetary rather than just national/regional, is excellent. > From that post: > > """ > 5.1 Gibberish > > When going from the original 2-byte unicode (around version 3?) to the > one having supplemental planes, the unicode consortium added blocks > such as > > * Egyptian hieroglyphs > * Cuneiform > * Shavian > * Deseret > * Mahjong > * Klingon > > To me (a layman) it looks unprofessional ? as though they are playing > games ? that billions of computing devices, each having billions of > storage words should have their storage wasted on blocks such as > these. > """ > > The shift from Unicode as a 16-bit code to having multiple planes came > in with Unicode 2.0, but the various blocks were assigned separately: > * Egyptian hieroglyphs: Unicode 5.2 > * Cuneiform: Unicode 5.0 > * Shavian: Unicode 4.0 > * Deseret: Unicode 3.1 > * Mahjong Tiles: Unicode 5.1 > * Klingon: Not part of any current standard You should add emoticons, but not call them or the above 'gibberish'. I think that this part of your post is more 'unprofessional' than the character blocks. It is very jarring and seems contrary to your main point. > However, I don't think historians will appreciate you calling all of > these "gibberish". To adequately describe and discuss old texts > without these Unicode blocks, we'd have to either do everything with > images, or craft some kind of reversible transliteration system and > have dedicated software to render the texts on screen. Instead, what > we have is a well-known and standardized system for transliterating > all of these into numbers (code points), and rendering them becomes a > simple matter of installing an appropriate font. > > Also, how does assigning meanings to codepoints "waste storage"? As > soon as Unicode 2.0 hit and 16-bit code units stopped being > sufficient, everyone needed to allocate storage - either 32 bits per > character, or some other system - and the fact that some codepoints > were unassigned had absolutely no impact on that. This is decidedly > NOT unprofessional, and it's not wasteful either. I agree. -- Terry Jan Reedy From rustompmody at gmail.com Thu Feb 26 12:08:24 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 26 Feb 2015 09:08:24 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: <0c1f4147-2d5d-4fa6-afb9-2275d878e2c1@googlegroups.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <0c1f4147-2d5d-4fa6-afb9-2275d878e2c1@googlegroups.com> Message-ID: On Thursday, February 26, 2015 at 10:16:11 PM UTC+5:30, Sam Raker wrote: > I'm 100% in favor of expanding Unicode until the sun goes dark. Doing so helps solve the problems affecting speakers of "underserved" languages--access and language preservation. Speakers of Mongolian, Cherokee, Georgian, etc. all deserve to be able to interact with technology in their native languages as much as we speakers of ASCII-friendly languages do. Unicode support also makes writing papers on, dictionaries of, and new texts in such languages much easier, which helps the fight against language extinction, which is a sadly pressing issue. Agreed -- Correcting the inequities caused by ASCII-bias is a good thing. In fact the whole point of my post was to say just that by carving out and focussing on a 'universal' subset of unicode that is considerably larger than ASCII but smaller than unicode, we stand to reduce ASCII-bias. As also other posts like http://blog.languager.org/2014/04/unicoded-python.html http://blog.languager.org/2014/05/unicode-in-haskell-source.html However my example listed > > * Egyptian hieroglyphs > > * Cuneiform > > * Shavian > > * Deseret > > * Mahjong > > * Klingon Ok Chris has corrected me re. Klingon-in-unicode. So lets drop that. Of the others which do you thing is in 'underserved' category? More generally which of http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Supplementary_Multilingual_Plane are underserved? From tjreedy at udel.edu Thu Feb 26 12:11:36 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Feb 2015 12:11:36 -0500 Subject: Installing PIL without internet access In-Reply-To: References: Message-ID: On 2/26/2015 10:23 AM, Larry Martell wrote: > I have a host that has no access to the internet and I need to install > PIL on it. I have an identical host that is on the internet and I have > installed it there (with pip). Is there a way I can copy files from > the connected host to a flash drive and then copy them to the > unconnected host and have PIL working there? Which files would I copy > for that? > > This is on CentOS 6.5, python 2.7 On Windows, I would look in python27/Lib/site-packages for PIL and pil-dist-info directories and copy. look in python27/script for pil*.py and copy -- Terry Jan Reedy From tjreedy at udel.edu Thu Feb 26 12:16:58 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Feb 2015 12:16:58 -0500 Subject: Parallelization of Python on GPU? In-Reply-To: <1424963166.30927.73.camel@gmail.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> Message-ID: On 2/26/2015 10:06 AM, Jason Swails wrote: > On Thu, 2015-02-26 at 14:02 +1100, Steven D'Aprano wrote: >> John Ladasky wrote: >> >> >>> What I would REALLY like to do is to take advantage of my GPU. >> >> I can't help you with that, but I would like to point out that GPUs >> typically don't support IEE-754 maths, which means that while they are >> likely significantly faster, they're also likely significantly less >> accurate. Any any two different brands/models of GPU are likely to give >> different results. (Possibly not *very* different, but considering the mess >> that floating point maths was prior to IEEE-754, possibly *very* different.) > > This hasn't been true in NVidia GPUs manufactured since ca. 2008. > >> Personally, I wouldn't trust GPU floating point for serious work. Maybe for >> quick and dirty exploration of the data, but I'd then want to repeat any >> calculations using the main CPU before using the numbers anywhere :-) > > There is a *huge* dash toward GPU computing in the scientific computing > sector. Since I started as a graduate student in computational > chemistry/physics in 2008, I watched as state-of-the-art supercomputers > running tens of thousands to hundreds of thousands of cores were > overtaken in performance by a $500 GPU (today the GTX 780 or 980) you > can put in a desktop. I went from running all of my calculations on a > CPU cluster in 2009 to running 90% of my calculations on a GPU by the > time I graduated in 2013... and for people without as ready access to > supercomputers as myself the move was even more pronounced. > > This work is very serious, and numerical precision is typically of > immense importance. See, e.g., > http://www.sciencedirect.com/science/article/pii/S0010465512003098 and > http://pubs.acs.org/doi/abs/10.1021/ct400314y > > In our software, we can run simulations on a GPU or a CPU and the > results are *literally* indistinguishable. The transition to GPUs was > accompanied by a series of studies that investigated precisely your > concerns... we would never have started using GPUs if we didn't trust > GPU numbers as much as we did from the CPU. > > And NVidia is embracing this revolution (obviously) -- they are putting > a lot of time, effort, and money into ensuring the success of GPU high > performance computing. It is here to stay in the immediate future, and > refusing to use the technology will leave those that *could* benefit > from it at a severe disadvantage. (That said, GPUs aren't good at > everything, and CPUs are also here to stay.) > > And GPU performance gains are outpacing CPU performance gains -- I've > seen about two orders of magnitude improvement in computational > throughput over the past 6 years through the introduction of GPU > computing and improvements in GPU hardware. Thanks for the update. -- Terry Jan Reedy From rosuav at gmail.com Thu Feb 26 12:29:28 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 04:29:28 +1100 Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: On Fri, Feb 27, 2015 at 4:02 AM, Terry Reedy wrote: > On 2/26/2015 8:24 AM, Chris Angelico wrote: >> >> On Thu, Feb 26, 2015 at 11:40 PM, Rustom Mody >> wrote: >>> >>> Wrote something up on why we should stop using ASCII: >>> http://blog.languager.org/2015/02/universal-unicode.html > > > I think that the main point of the post, that many Unicode chars are truly > planetary rather than just national/regional, is excellent. Agreed. Like you, though, I take exception at the "Gibberish" section. Unicode offers us a number of types of character needed by linguists: 1) Letters[1] common to many languages, such as the unadorned Latin and Cyrillic letters 2) Letters specific to one or very few languages, such as the Turkish dotless i 3) Diacritical marks, ready to be combined with various letters 4) Precomposed forms of various common "letter with diacritical" combinations 5) Other precomposed forms, eg ligatures and Hangul syllables 6) Symbols, punctuation, and various other marks 7) Spacing of various widths and attributes Apart from #4 and #5, which could be avoided by using the decomposed forms everywhere, each of these character types is vital. You can't typeset a document without being able to adequately represent every part of it. Then there are additional characters that aren't strictly necessary, but are extremely convenient, such as the emoticon sections. You can talk in text and still put in a nice little picture of a globe, or the monkey-no-evil set, etc. Most of these characters - in fact, all except #2 and maybe a few of the diacritical marks - are used in multiple places/languages. Unicode isn't about taking everyone's separate character sets and numbering them all so we can reference characters from anywhere; if you wanted that, you'd be much better off with something that lets you specify a code page in 16 bits and a character in 8, which is roughly the same size as Unicode anyway. What we have is, instead, a system that brings them all together - LATIN SMALL LETTER A is U+0061 no matter whether it's being used to write English, French, Malaysian, Turkish, Croatian, Vietnamese, or Icelandic text. Unicode is truly planetary. ChrisA [1] I use the word "letter" loosely here; Chinese and Japanese don't have a concept of letters as such, but their glyphs are still represented. From john_ladasky at sbcglobal.net Thu Feb 26 12:34:16 2015 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 26 Feb 2015 09:34:16 -0800 (PST) Subject: Parallelization of Python on GPU? In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: On Thursday, February 26, 2015 at 8:41:26 AM UTC-8, Sturla Molden wrote: > If you are doing SVM regression with scikit-learn you are using libSVM. > There is a CUDA accelerated version of this C library here: > http://mklab.iti.gr/project/GPU-LIBSVM > > You can presumably reuse the wrapping code from scikit-learn. > > Sturla Hi Sturla, I recognize your name from the scikit-learn mailing list. If you look a few posts above yours in this thread, I am aware of gpu-libsvm. I don't know if I'm up to the task of reusing the scikit-learn wrapping code, but I am giving that option some serious thought. It isn't clear to me that gpu-libsvm can handle both SVM and SVR, and I have need of both algorithms. My training data sets are around 5000 vectors long. IF that graph on the gpu-libsvm web page is any indication of what I can expect from my own data (I note that they didn't specify the GPU card they're using), I might realize a 20x increase in speed. From jason.swails at gmail.com Thu Feb 26 12:48:03 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 26 Feb 2015 12:48:03 -0500 Subject: Parallelization of Python on GPU? In-Reply-To: <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> Message-ID: <1424972883.30927.138.camel@gmail.com> On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote: > GPU computing is great if you have the following: > > 1. Your data structures are arrays floating point numbers. It actually works equally great, if not better, for integers. > 2. You have a data-parallel problem. This is the biggest one, IMO. ^^^ > 3. You are happy with single precision. NVidia GPUs have double-precision maths in hardware since compute capability 1.2 (GTX 280). That's ca. 2008. In optimized CPU code, you still get ~50% benefit going from double to single precision (it's rarely ever that high, but 20-30% is commonplace in my experience of optimized code). It's admittedly a bigger hit on most GPUs, but there are ways to work around it (e.g., fixed precision), and you can still do double precision work where it's needed. One of the articles I linked previously demonstrates that a hybrid precision model (based on fixed precision) provides exactly the same numerical stability as double precision (which is much better than pure single precision) for that application. Double precision can often be avoided in many parts of a calculation, using it only where those bits matter (like accumulators with potentially small contributions, subtractions of two numbers of similar magnitude, etc.). > 4. You have time to code erything in CUDA or OpenCL. This is the second biggest one, IMO. ^^^ > 5. You have enough video RAM to store your data. Again, it can be worked around, but the frequent GPU->CPU xfers involved if you can't fit everything on the GPU can be painstaking to limit its potentially devastating effects on performance. > > For Python the easiest solution is to use Numba Pro. Agreed, although I've never actually tried PyCUDA before... All the best, Jason From rustompmody at gmail.com Thu Feb 26 12:59:05 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 26 Feb 2015 09:59:05 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> On Thursday, February 26, 2015 at 10:33:44 PM UTC+5:30, Terry Reedy wrote: > On 2/26/2015 8:24 AM, Chris Angelico wrote: > > On Thu, Feb 26, 2015 at 11:40 PM, Rustom Mody wrote: > >> Wrote something up on why we should stop using ASCII: > >> http://blog.languager.org/2015/02/universal-unicode.html > > I think that the main point of the post, that many Unicode chars are > truly planetary rather than just national/regional, is excellent. > > > From that post: > > > > """ > > 5.1 Gibberish > > > > When going from the original 2-byte unicode (around version 3?) to the > > one having supplemental planes, the unicode consortium added blocks > > such as > > > > * Egyptian hieroglyphs > > * Cuneiform > > * Shavian > > * Deseret > > * Mahjong > > * Klingon > > > > To me (a layman) it looks unprofessional ? as though they are playing > > games ? that billions of computing devices, each having billions of > > storage words should have their storage wasted on blocks such as > > these. > > """ > > > > The shift from Unicode as a 16-bit code to having multiple planes came > > in with Unicode 2.0, but the various blocks were assigned separately: > > * Egyptian hieroglyphs: Unicode 5.2 > > * Cuneiform: Unicode 5.0 > > * Shavian: Unicode 4.0 > > * Deseret: Unicode 3.1 > > * Mahjong Tiles: Unicode 5.1 > > * Klingon: Not part of any current standard > > You should add emoticons, but not call them or the above 'gibberish'. Emoticons (or is it emoji) seems to have some (regional?) takeup?? Dunno? In any case I'd like to stay clear of political(izable) questions > I think that this part of your post is more 'unprofessional' than the > character blocks. It is very jarring and seems contrary to your main point. Ok I need a word for 1. I have no need for this 2. 99.9% of the (living) on this planet also have no need for this > > > However, I don't think historians will appreciate you calling all of > > these "gibberish". To adequately describe and discuss old texts > > without these Unicode blocks, we'd have to either do everything with > > images, or craft some kind of reversible transliteration system and > > have dedicated software to render the texts on screen. Instead, what > > we have is a well-known and standardized system for transliterating > > all of these into numbers (code points), and rendering them becomes a > > simple matter of installing an appropriate font. > > > > Also, how does assigning meanings to codepoints "waste storage"? As > > soon as Unicode 2.0 hit and 16-bit code units stopped being > > sufficient, everyone needed to allocate storage - either 32 bits per > > character, or some other system - and the fact that some codepoints > > were unassigned had absolutely no impact on that. This is decidedly > > NOT unprofessional, and it's not wasteful either. > > I agree. I clearly am more enthusiastic than knowledgeable about unicode. But I know my basic CS well enough (as I am sure you and Chris also do) So I dont get how 4 bytes is not more expensive than 2. Yeah I know you can squeeze a unicode char into 3 bytes or even 21 bits You could use a clever representation like UTF-8 or FSR. But I dont see how you can get out of this that full-unicode costs more than exclusive BMP. eg consider the case of 32 vs 64 bit executables. The 64 bit executable is generally larger than the 32 bit one Now consider the case of a machine that has say 2GB RAM and a 64-bit processor. You could -- I think -- make a reasonable case that all those all-zero hi-address-words are 'waste'. And youve got the general sense best so far: > I think that the main point of the post, that many Unicode chars are > truly planetary rather than just national/regional, And if the general tone/tenor of what I have written is probably not getting across by some words (like 'gibberish'?) so I'll try and reword. However let me try and clarify that the whole of section 5 is 'iffy' with 5.1 being only more extreme. Ive not written these in because the point of that post is not to criticise unicode but to highlight the universal(isable) parts. Still if I were to expand on the criticisms here are some examples: Math-Greek: Consider the math-alpha block http://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode#Mathematical_Alphanumeric_Symbols_block Now imagine a beginning student not getting the difference between font, glyph, character. To me this block represents this same error cast into concrete and dignified by the (supposed) authority of the unicode consortium. There are probably dozens of other such stupidities like distinguishing kelvin ? from latin K as if that is the business of the unicode consortium My real reservations about unicode come from their work in areas that I happen to know something about Music: To put music simply as a few mostly-meaningless 'dingbats' like ? ? ? is perhaps ok However all this stuff http://xahlee.info/comp/unicode_music_symbols.html makes no sense (to me) given that music (ie standard western music written in staff notation) is inherently 2 dimensional -- multi-voiced, multi-staff, chordal Sanskrit/Devanagari: Consists of bogus letters that dont exist in devanagari The letter ? (0904) is found here http://unicode.org/charts/PDF/U0900.pdf But not here http://en.wikipedia.org/wiki/Devanagari#Vowels So I call it bogus-devanagari Contrariwise an important letter in vedic pronunciation the double-udatta is missing http://list.indology.info/pipermail/indology_list.indology.info/2000-April/021070.html All of which adds up to the impression that the unicode consortium occasionally fails to do due diligence In any case all of the above is contrary to /irrelevant to my post which is about identifying the more universal parts of unicode From jaykim.huijae at gmail.com Thu Feb 26 12:59:45 2015 From: jaykim.huijae at gmail.com (jaykim.huijae at gmail.com) Date: Thu, 26 Feb 2015 09:59:45 -0800 (PST) Subject: Gaussian process regression Message-ID: <5d2cd3fd-9d63-4e79-af1c-965d50145801@googlegroups.com> Hi, I am trying to use Gaussian process regression for Near Infrared spectra. I have reference data(spectra), concentrations of reference data and sample data, and I am trying to predict concentrations of sample data. Here is my code. from sklearn.gaussian_process import GaussianProcess gp = GaussianProcess() gp.fit(reference, concentration) concentration_pred = gp.predict(sample) The results always gave me the same concentration even though I used different sample data. When I used some parts of reference data as sample data, it predicted concentration well. But whenever I use different data than reference data, it always gave me the same concentration. Can I get some help with this problem? What am I doing wrong? I would appreciate any help. Thanks, Jay From af300wsm at gmail.com Thu Feb 26 13:25:41 2015 From: af300wsm at gmail.com (af300wsm at gmail.com) Date: Thu, 26 Feb 2015 10:25:41 -0800 (PST) Subject: Building C++ modules for python using GNU autotools, automake, whatever In-Reply-To: References: <364c8f3d-808e-4513-9041-6d8cdfdc42b7@googlegroups.com> Message-ID: <3ebf8451-9e58-4ed0-8d89-47dd98e59b80@googlegroups.com> On Thursday, February 26, 2015 at 9:35:12 AM UTC-7, Jason Swails wrote: > On Thu, 2015-02-26 at 07:57 -0800, af300wsm wrote: > > Hi, > > > > I'm a complete neophyte to the whole use of GNU > > autotools/automake/auto... . (I'm not sure what it should be called > > anymore.) Regardless, I'm porting a library project, for which I'm a > > team member, to using this toolset for building in Linux. I'm to the > > point now of writing the Makefile.am file for the actual library. > > (There are several other static libraries compiled first that are > > sucked into this shared object file.) > > > > I found some references here: > > http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Python.html, which seemed to be just what I was after. However, I've got a big question about a file named "module.la" instead of "module.so" which is what we compile it to now. > > I certainly hope module.la is not what it gets compiled to. Open it up > with a text editor :). It's just basically a description of the library Fascinating! This is all new territory for me. I'm used these tools for a number of years, of course, as I've run "./configure && make && make install" many times. Now things are starting to make more sense. > that libtool makes use of. In the projects that I build, the .la files > are all associated with a .a archive or a .so (/.dylib for Macs). > Obviously, static archives won't work for Python (and, in particular, I > believe you need to compile all of the objects as position independent > code, so you need to make sure the appropriate PIC flag is given to the > compiler... for g++ that would be -fPIC). We are compiling all of our code with -fPIC. I looked over the final build line and I see that a module.so was placed in .libs. I looked in that directory and actually the module is named "module.so.0.0.0" and there is a symbolic link "module.so" which points to that. This is cool stuff. Thanks for the clarification on things. From cybervigilante at gmail.com Thu Feb 26 14:04:48 2015 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 26 Feb 2015 11:04:48 -0800 (PST) Subject: ANN: Wing IDE 5.1.2 released In-Reply-To: References: Message-ID: Hey, can I run Py 2.7 and 3.4 side by side without a lot of hassle, using Wing? I run both since I'm migranting and so far the free IDEs just seem to choke on that. From jaiprakashsingh213 at gmail.com Thu Feb 26 14:14:50 2015 From: jaiprakashsingh213 at gmail.com (Jai) Date: Thu, 26 Feb 2015 11:14:50 -0800 (PST) Subject: requesting you all to please guide me , which tutorials is best to learn redis database Message-ID: hello all, i want to learn redis database and its use via python , please guide me which tutorials i should be study, so that i can learn it in good way I search this on google but i am little confuse, so please help me thank you jai From larry.martell at gmail.com Thu Feb 26 14:22:41 2015 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 26 Feb 2015 14:22:41 -0500 Subject: Installing PIL without internet access In-Reply-To: <54EF5079.7080402@mrabarnett.plus.com> References: <54EF5079.7080402@mrabarnett.plus.com> Message-ID: On Thu, Feb 26, 2015 at 11:57 AM, MRAB wrote: > On 2015-02-26 15:23, Larry Martell wrote: >> >> I have a host that has no access to the internet and I need to install >> PIL on it. I have an identical host that is on the internet and I have >> installed it there (with pip). Is there a way I can copy files from >> the connected host to a flash drive and then copy them to the >> unconnected host and have PIL working there? Which files would I copy >> for that? >> >> This is on CentOS 6.5, python 2.7 >> > Have a look here: > > https://pip.pypa.io/en/latest/reference/pip_install.html#pip-install-options > > It says that you can install from a downloaded file, e.g.: > > pip install ./downloads/SomePackage-1.0.4.tar.gz Thanks for the reply. This is very useful info. But I have another issue I didn't mention. The system python is 2.6, but I need the 2.7 version. So anything I install with pip will get installed to 2.6. To get around that on my connected hosts I've done: easy_install-2.7 pip and then I install with pip2.7. But this unconnected host doesn't have easy_install-2.7, so I'd have to figure out how to get that first. I think it will work if I just copy /usr/lib64/python2.7/site-packages/PIL. That worked on a test system I tried it on. I'll try on the real system tonight. From ben+python at benfinney.id.au Thu Feb 26 14:24:05 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 27 Feb 2015 06:24:05 +1100 Subject: Python Worst Practices References: <85fv9t5zen.fsf@benfinney.id.au> <54EE88B2.1020008@gmail.com> <85d24x48na.fsf@benfinney.id.au> <0583sb-j98.ln1@esprimo.zbmc.eu> Message-ID: <85pp8w3296.fsf@benfinney.id.au> cl at isbd.net writes: > Ben Finney wrote: > > My request at the moment, though, is not for people to change what's > > on their slides; rather, if they want people to retrieve them, the > > slides should be downloadable easily (i.e. without a web app, > > without a registration to some specific site). > > ... and having downloaded them what do you view them with if they're > not plain text? Again, I was not the one asking for plain text. So I don't really understand why you ask me that. But, here goes: Presentations documents, the overwhelming majority, are in a very small number of formats. If they're PDF: any PDF viewer . If they're a format produced by some widespread presentation tool: LibreOffice Impress . Why do you ask? -- \ ?The way to build large Python applications is to componentize | `\ and loosely-couple the hell out of everything.? ?Aahz | _o__) | Ben Finney From alister.nospam.ware at ntlworld.com Thu Feb 26 14:32:35 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Thu, 26 Feb 2015 19:32:35 +0000 (UTC) Subject: Python Worst Practices References: <54EE3E6A.5010204@mrabarnett.plus.com> Message-ID: On Wed, 25 Feb 2015 23:34:29 +0000, MRAB wrote: > On 2015-02-25 22:59, Joel Goldstick wrote: > > On Wed, Feb 25, 2015 at 4:28 PM, MRAB > > wrote: > > > On 2015-02-25 20:45, Mark Lawrence wrote: > > >> > > >> http://www.slideshare.net/pydanny/python-worst-practices > > >> > > >> Any that should be added to this list? Any that be removed as not > that > > >> bad? > > >> > > > We don't have numeric ZIP codes in the UK, but the entire world has > > > numeric telephone numbers, so that might be a better example of > > > numbers that aren't really numbers. > > > > US zip codes get messed up with ints because many have a leading > > zero. > > I use strings > > Telephone numbers can also start with zero. unless you are performing maths on it data that is made up of numbers (zip code, tel number, house number etc) is still only text & should be stored as a string. > > > > Numeric dates can be ambiguous: dd/mm/yyyy or mm/dd/yyyy? The ISO > > > standard is clearer: yyyy-mm-dd. > > > > > > Handling text: "Unicode sandwich". > > > > > > UTF-8 is better than legacy encodings. > > > -- To save a single life is better than to build a seven story pagoda. From simon+python at bleah.co.uk Thu Feb 26 14:36:02 2015 From: simon+python at bleah.co.uk (Simon Ward) Date: Thu, 26 Feb 2015 19:36:02 +0000 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <2f8fd253-6abb-455b-bf1f-e7e3af87239d@email.android.com> On 25 February 2015 21:24:37 GMT+00:00, Chris Angelico wrote: >On Thu, Feb 26, 2015 at 7:45 AM, Mark Lawrence > wrote: >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not >that bad? > >Remove the complaint about id. It's an extremely useful variable name, >and you hardly ever need the function. You can add one character and avoid the conflict with "id_" and not require anyone else maintaining the code to think about it. As rare as the conflict is, I think the ease of avoiding it makes the extra character a practical defensive technique. I agree it is not a worst case. Simon From jvarghese at quixey.com Thu Feb 26 14:37:45 2015 From: jvarghese at quixey.com (jvarghese at quixey.com) Date: Thu, 26 Feb 2015 11:37:45 -0800 (PST) Subject: splunk_handler and logbook In-Reply-To: References: Message-ID: I got the solution, Use RedirectLoggingHandler, to redirect the logs to logbook, from logging import getLogger mylog = getLogger('My Log') from splunk_handler import SplunkHandler splunk = SplunkHandler( ... host='', ... port='', ... username='', ... password='', ... index='', ... verify=, ... source="" ... ) from logbook.compat import RedirectLoggingHandler mylog.addHandler(RedirectLoggingHandler()) mylog.addHandler(splunk) ...... From wrw at mac.com Thu Feb 26 14:42:55 2015 From: wrw at mac.com (William Ray Wing) Date: Thu, 26 Feb 2015 14:42:55 -0500 Subject: ANN: Wing IDE 5.1.2 released In-Reply-To: References: Message-ID: > On Feb 26, 2015, at 2:04 PM, Jim Mooney wrote: > > Hey, can I run Py 2.7 and 3.4 side by side without a lot of hassle, using Wing? I run both since I'm migranting and so far the free IDEs just seem to choke on that. > -- > https://mail.python.org/mailman/listinfo/python-list I assume you just mean that you would like to have different Python projects that open in Wing with the correct associated version of Python. Yes, you can specify a python executable in the Project Properties - Environment tab. Click on the ?Custom" button in the Python Executable entry and enter the path to the version of Python you want. If this isn?t what you are after, let us know. -Bill PS: I?ve found that the Wing e-mail support is VERY responsive. No relation, just a happy user. From ian.g.kelly at gmail.com Thu Feb 26 14:54:08 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 26 Feb 2015 12:54:08 -0700 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: On Feb 26, 2015 4:00 AM, "Cem Karan" wrote: > > > On Feb 26, 2015, at 12:36 AM, Gregory Ewing wrote: > > > Cem Karan wrote: > >> I think I see what you're talking about now. Does WeakMethod > >> (https://docs.python.org/3/library/weakref.html#weakref.WeakMethod) solve > >> this problem? > > > > Yes, that looks like it would work. > > > Cool! Sometimes I wonder whether anybody reads my posts. I suggested a solution involving WeakMethod four days ago that additionally extends the concept to non-method callbacks (requiring a small amount of extra effort from the client in those cases, but I think that is unavoidable. There is no way that the framework can determine the appropriate lifetime for a closure-based callback.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Thu Feb 26 14:59:25 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 26 Feb 2015 11:59:25 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <26b84544-f24f-4298-b631-bb0c2f35e8d1@googlegroups.com> On Thursday, February 26, 2015 at 10:33:44 PM UTC+5:30, Terry Reedy wrote: > You should add emoticons, but not call them or the above 'gibberish'. Done -- and of course not under gibberish. I dont really know much how emoji are used but I understand they are. JFTR I consider it necessary to be respectful to all (living) people. For that matter even dead people(s) - no need to be disrespectful to the egyptians who created the hieroglyphs or the sumerians who wrote cuneiform. I only find it crosses a line when the 2 millenia dead creations are made to take the space of the living. Chris wrote: > * Klingon: Not part of any current standard Thanks Removed. From ethan at stoneleaf.us Thu Feb 26 15:00:35 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 26 Feb 2015 12:00:35 -0800 Subject: Design thought for callbacks In-Reply-To: References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> Message-ID: <54EF7B63.3050009@stoneleaf.us> On 02/26/2015 11:54 AM, Ian Kelly wrote: > Sometimes I wonder whether anybody reads my posts. It's entirely possible the OP wasn't ready to understand your solution four days ago, but two days later the OP was. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From invalid at invalid.invalid Thu Feb 26 15:04:14 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Thu, 26 Feb 2015 20:04:14 +0000 (UTC) Subject: Python Worst Practices References: <54EE3E6A.5010204@mrabarnett.plus.com> Message-ID: On 2015-02-26, alister wrote: > On Wed, 25 Feb 2015 23:34:29 +0000, MRAB wrote: >> On 2015-02-25 22:59, Joel Goldstick wrote: >>> On Wed, Feb 25, 2015 at 4:28 PM, MRAB wrote: >>>> On 2015-02-25 20:45, Mark Lawrence wrote: >>>>> >>>>> http://www.slideshare.net/pydanny/python-worst-practices >>>>> >>>>> Any that should be added to this list? Any that be removed as not >>>>> that bad? >>>> >>>> We don't have numeric ZIP codes in the UK, but the entire world has >>>> numeric telephone numbers, so that might be a better example of >>>> numbers that aren't really numbers. >>> >>> US zip codes get messed up with ints because many have a leading >>> zero. >>> I use strings I should hope so, because US zip codes can also contain a hyphen. >> Telephone numbers can also start with zero. > > unless you are performing maths on it data that is made up of numbers > (zip code, tel number, house number etc) is still only text & should be > stored as a string. And if you _are_ performing maths on postal codes, telephone numbers and house numbers, something is seriously wrong and it probably doesn't matter how you represent things. -- Grant Edwards grant.b.edwards Yow! I am covered with at pure vegetable oil and I am gmail.com writing a best seller! From simon+python at bleah.co.uk Thu Feb 26 15:10:28 2015 From: simon+python at bleah.co.uk (Simon Ward) Date: Thu, 26 Feb 2015 20:10:28 +0000 Subject: Python Worst Practices In-Reply-To: <85zj814jmb.fsf@benfinney.id.au> References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> Message-ID: <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> On 26 February 2015 00:11:24 GMT+00:00, Ben Finney wrote: >> Yes, but my point is: You shouldn't need to rebind those names (or >> have names "true" and "false" for 0 and 1). > >That's not what you asked, though. You asked ?When would 0 mean true >and >1 mean false?? My answer: in all Unix shell contexts. > >> Instead, use "success" and "failure". > >You'd better borrow the time machine and tell the creators of Unix. The >meme is already established for decades now. 0 = success and non-zero = failure is the meme established, rather than 0 = true, non-zero = false. It's not just used by UNIX, and is not necessarily defined by the shell either (bash was mentioned elsewhere in the thread). There is probably a system that pre-dates UNIX that I uses/used this too, but I don't know. C stdlib defines EXIT_SUCCESS = 0, yet C99 stdbool.h defines false = 0. That shells handle 0 as true and non-zero as false probably stems from this (or similar in older languages). The " true" command is defined to have an exit status of 0, and "false" an exit status of 1. The value is better thought of an error level, where 0 is no error and non-zero is some error. The AmigaOS shell conventionally takes this further with higher values indicating more critical errors, there's even a "failat N" command that means exit the script if the error level is higher than N. None of the above is a good reason to use error *or* success return values in Python--use exceptions!--but may be encountered when running other processes. Simon From wxjmfauth at gmail.com Thu Feb 26 15:20:32 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 26 Feb 2015 12:20:32 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> Message-ID: <5b002d84-3ad8-4a4a-8852-69ed93b45ff3@googlegroups.com> Le jeudi 26 f?vrier 2015 18:59:24 UTC+1, Rustom Mody a ?crit?: > > ...To me this block represents this same error cast into concrete and > dignified by the (supposed) authority of the unicode consortium. > Unicode does not prescribe, it registrates. Eg. The inclusion of U+1E9E, 'LATIN CAPITAL LETTER SHARP S' has been officialy proposed by the "German Federal Government". (I have a pdf copy somewhere). From breamoreboy at yahoo.co.uk Thu Feb 26 15:54:08 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 26 Feb 2015 20:54:08 +0000 Subject: Windows permission error, 64 bit, psycopg2, python 3.4.2 In-Reply-To: <6a27dcaa-4652-4dc4-8c74-fd0a68ebb51a@googlegroups.com> References: <6a27dcaa-4652-4dc4-8c74-fd0a68ebb51a@googlegroups.com> Message-ID: On 26/02/2015 15:10, Malik Rumi wrote: > I am one of those struggling with compile issues with python on 64 bit windows. I have not been able to get the solutions mentioned on Stack Overflow to work because installing Windows SDK 7.1 fails for me. > > So I stumbled across a precompiled psycopg2, and that reported that it worked, but then I got two permission errors. Then I read that this was a bug in python (issue 14252) that had been fixed, but I don't think this is the same error. That one specifically refers to subprocess.py and I don't have that in my traceback. I have v3.4.2. On top of everything else, despite requesting a new password, all I get from the big tracker is 'invalid login'. > > In any event, running "import psycopg2" returns 'import error, no module named psycopg2'. > > > Microsoft Windows [Version 6.3.9600] > (c) 2013 Microsoft Corporation. All rights reserved. > > C:\Users\Semantic>pip install git+https://github.com/nwcell/psycopg2-windows.git > @win64-py34#egg=psycopg2 > Downloading/unpacking psycopg2 from git+https://github.com/nwcell/psycopg2-windo > ws.git at win64-py34 > Cloning https://github.com/nwcell/psycopg2-windows.git (to win64-py34) to c:\u > sers\semantic\appdata\local\temp\pip_build_semantic\psycopg2 > Running setup.py (path:C:\Users\Semantic\AppData\Local\Temp\pip_build_Semantic > \psycopg2\setup.py) egg_info for package psycopg2 > C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt > ion: 'summary' > warnings.warn(msg) > > Installing collected packages: psycopg2 > Running setup.py install for psycopg2 > C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt > ion: 'summary' > warnings.warn(msg) > > Successfully installed psycopg2 > Cleaning up... > Exception: > Traceback (most recent call last): > File "C:\Python34\lib\shutil.py", line 370, in _rmtree_unsafe > os.unlink(fullname) > PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Semantic\\AppData\\L > ocal\\Temp\\pip_build_Semantic\\psycopg2\\.git\\objects\\pack\\pack-be4d3da4a06b > 4c9ec4c06040dbf6685eeccca068.idx' > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "C:\Python34\lib\site-packages\pip\basecommand.py", line 122, in main > status = self.run(options, args) > File "C:\Python34\lib\site-packages\pip\commands\install.py", line 302, in run > > requirement_set.cleanup_files(bundle=self.bundle) > File "C:\Python34\lib\site-packages\pip\req.py", line 1333, in cleanup_files > rmtree(dir) > File "C:\Python34\lib\site-packages\pip\util.py", line 43, in rmtree > onerror=rmtree_errorhandler) > File "C:\Python34\lib\shutil.py", line 477, in rmtree > return _rmtree_unsafe(path, onerror) > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > _rmtree_unsafe(fullname, onerror) > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > _rmtree_unsafe(fullname, onerror) > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > _rmtree_unsafe(fullname, onerror) > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > _rmtree_unsafe(fullname, onerror) > File "C:\Python34\lib\shutil.py", line 372, in _rmtree_unsafe > onerror(os.unlink, fullname, sys.exc_info()) > File "C:\Python34\lib\site-packages\pip\util.py", line 53, in rmtree_errorhand > ler > (exctype is PermissionError and value.args[3] == 5) #python3.3 > IndexError: tuple index out of range > The above clearly shows "Successfully installed psycopg2" and that it's a permission error on cleanup that's gone wrong, so what is there to report on the bug tracker? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From sturla.molden at gmail.com Thu Feb 26 15:54:21 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 26 Feb 2015 21:54:21 +0100 Subject: Parallelization of Python on GPU? In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: On 26/02/15 18:34, John Ladasky wrote: > Hi Sturla, I recognize your name from the scikit-learn mailing list. > > If you look a few posts above yours in this thread, I am aware of gpu-libsvm. I don't know if I'm up to the task of reusing the scikit-learn wrapping code, but I am giving that option some serious thought. It isn't clear to me that gpu-libsvm can handle both SVM and SVR, and I have need of both algorithms. > > My training data sets are around 5000 vectors long. IF that graph on the gpu-libsvm web page is any indication of what I can expect from my own data (I note that they didn't specify the GPU card they're using), I might realize a 20x increase in speed. A GPU is a "floating point monster", not a CPU. It is not designed to run things like CPython. It is also only designed to run threads in parallel on its cores, not processes. And as you know, in Python there is something called GIL. Further the GPU has hard-wired fine-grained load scheduling for data-parallel problems (e.g. matrix multiplication for vertex processing in 3D graphics). It is not like a thread on a GPU is comparable to a thread on a CPU. It is more like a parallel work queue, with the kind of abstraction you find in Apple's GCD. I don't think it really doable to make something like CPython run with thousands of parallel instances on a GPU. A GPU is not designed for that. A GPU is great if you can pass millions of floating point vectors as items to the work queue, with a tiny amount of computation per item. It would be crippled if you passed a thousand CPython interpreters and expect them to do a lot of work. Also, as it is libSVM that does the math in you case, you need to get libSVM to run on the GPU, not CPython. In most cases the best hardware for parallel scientific computing (taking economy and flexibility into account) is a Linux cluster which supports MPI. You can then use mpi4py or Cython to use MPI from your Python code. Sturla From fomcl at yahoo.com Thu Feb 26 16:05:58 2015 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Thu, 26 Feb 2015 21:05:58 +0000 (UTC) Subject: Python Worst Practices In-Reply-To: <2f8fd253-6abb-455b-bf1f-e7e3af87239d@email.android.com> References: <2f8fd253-6abb-455b-bf1f-e7e3af87239d@email.android.com> Message-ID: <1974793549.294261.1424984758670.JavaMail.yahoo@mail.yahoo.com> ----- Original Message ----- > From: Simon Ward > To: > Cc: "python-list at python.org" > Sent: Thursday, February 26, 2015 8:36 PM > Subject: Re: Python Worst Practices > > > > On 25 February 2015 21:24:37 GMT+00:00, Chris Angelico > wrote: >> On Thu, Feb 26, 2015 at 7:45 AM, Mark Lawrence >> wrote: >>> http://www.slideshare.net/pydanny/python-worst-practices >>> >>> Any that should be added to this list? Any that be removed as not >> that bad? >> >> Remove the complaint about id. It's an extremely useful variable name, >> and you hardly ever need the function. > > You can add one character and avoid the conflict with "id_" and not > require anyone else maintaining the code to think about it. As rare as the > conflict is, I think the ease of avoiding it makes the extra character a > practical defensive technique. I agree it is not a worst case. > I sometimes do: import sys, functools if sys.version_info.major > 2: bytez = functools.partial(bytes, encoding="utf-8") else: bytez = bytes # nog encoding param in python 2. I bitez you when you shadow 'bytes' (I can't remember when I couldn't use the functools.partial object), though it often works. Much easier to use 'bytez' or 'bytes_'. It is annoying to 'unshadow' your code and confusing for others who might read your code. Albert-Jan From sturla.molden at gmail.com Thu Feb 26 16:10:42 2015 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 26 Feb 2015 22:10:42 +0100 Subject: Parallelization of Python on GPU? In-Reply-To: <1424972883.30927.138.camel@gmail.com> References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> <1424972883.30927.138.camel@gmail.com> Message-ID: On 26/02/15 18:48, Jason Swails wrote: > On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote: >> GPU computing is great if you have the following: >> >> 1. Your data structures are arrays floating point numbers. > > It actually works equally great, if not better, for integers. Right, but not complicated data structures with a lot of references or pointers. It requires data are laid out in regular arrays, and then it acts on these arrays in a data-parallel manner. It is designed to process vertices in parallel for computer graphics, and that is a limitation which is always there. It is not a CPU with 1024 cores. It is a "floating point monster" which can process 1024 vectors in parallel. You write a tiny kernel in a C-like language (CUDA, OpenCL) to process one vector, and then it will apply the kernel to all the vectors in an array of vectors. It is very comparable to how GLSL and Direct3D vertex and fragment shaders work. (The reason for which should be obvious.) The GPU is actually great for a lot of things in science, but it is not a CPU. The biggest mistake in the GPGPU hype is the idea that the GPU will behave like a CPU with many cores. Sturla From ray.cote at auricsystems.com Thu Feb 26 16:17:13 2015 From: ray.cote at auricsystems.com (Raymond Cote) Date: Thu, 26 Feb 2015 16:17:13 -0500 Subject: Fix for no module named _sysconfigdata while compiling Message-ID: <48D598E0-25DF-4D99-B9BF-27D1BA855A62@auricsystems.com> Thought I might help someone else address a problem I ran into this afternoon. While compiling Python 2.7.9 on CentOS 6, I received the error: no module named _sysconfigdata Googling found a number of other people having this problem ? but the other issues were all after the Python was installed ? not while building. In digging through their advice, I saw a number of them spoke about having multiple versions of Python installed. In my case, I already had a custom Python 2.7.3 installed on this machine ? and I was upgrading over it to Python 2.7.9. I found that renaming my custom /opt/python2.7 directory and then building the new release in the same directory, that the problem went away. Summary: Compiling Python 2.7.9 resulted in error: no module named _sysconfigdata while compiling. My configuration: ./configure --prefix=/opt/python2.7 --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /opt/python2.7/lib" make;make alt install Remove the existing /opt/python2.7 directory which had Python 2.7.3. Now all builds and installs properly. ?Ray -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ben+python at benfinney.id.au Thu Feb 26 16:23:34 2015 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 27 Feb 2015 08:23:34 +1100 Subject: Python Worst Practices References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> Message-ID: <85h9u82wq1.fsf@benfinney.id.au> Simon Ward writes: > On 26 February 2015 00:11:24 GMT+00:00, Ben Finney wrote: > >You'd better borrow the time machine and tell the creators of Unix. The > >meme is already established for decades now. > > 0 = success and non-zero = failure is the meme established, rather > than 0 = true, non-zero = false. That is not the case: the commands ?true? (returns value 0) and ?false? (returns value 1) are long established in Unix. So that *is* part of the meme I'm describing. > None of the above is a good reason to use error *or* success return > values in Python--use exceptions!--but may be encountered when running > other processes. Right. But likewise, don't deny that ?true == 0? and ?false == non-zero? has a wide acceptance in the programming community too. -- \ ?Program testing can be a very effective way to show the | `\ presence of bugs, but is hopelessly inadequate for showing | _o__) their absence.? ?Edsger W. Dijkstra | Ben Finney From malik.a.rumi at gmail.com Thu Feb 26 16:42:33 2015 From: malik.a.rumi at gmail.com (Malik Rumi) Date: Thu, 26 Feb 2015 13:42:33 -0800 (PST) Subject: Windows permission error, 64 bit, psycopg2, python 3.4.2 In-Reply-To: References: <6a27dcaa-4652-4dc4-8c74-fd0a68ebb51a@googlegroups.com> Message-ID: <7ab6bcf5-62bd-47f6-858f-81fb6f275c25@googlegroups.com> On Thursday, February 26, 2015 at 2:55:07 PM UTC-6, Mark Lawrence wrote: > On 26/02/2015 15:10, Malik Rumi wrote: > > I am one of those struggling with compile issues with python on 64 bit windows. I have not been able to get the solutions mentioned on Stack Overflow to work because installing Windows SDK 7.1 fails for me. > > > > So I stumbled across a precompiled psycopg2, and that reported that it worked, but then I got two permission errors. Then I read that this was a bug in python (issue 14252) that had been fixed, but I don't think this is the same error. That one specifically refers to subprocess.py and I don't have that in my traceback. I have v3.4.2. On top of everything else, despite requesting a new password, all I get from the big tracker is 'invalid login'. > > > > In any event, running "import psycopg2" returns 'import error, no module named psycopg2'. > > > > > > Microsoft Windows [Version 6.3.9600] > > (c) 2013 Microsoft Corporation. All rights reserved. > > > > C:\Users\Semantic>pip install git+https://github.com/nwcell/psycopg2-windows.git > > @win64-py34#egg=psycopg2 > > Downloading/unpacking psycopg2 from git+https://github.com/nwcell/psycopg2-windo > > ws.git at win64-py34 > > Cloning https://github.com/nwcell/psycopg2-windows.git (to win64-py34) to c:\u > > sers\semantic\appdata\local\temp\pip_build_semantic\psycopg2 > > Running setup.py (path:C:\Users\Semantic\AppData\Local\Temp\pip_build_Semantic > > \psycopg2\setup.py) egg_info for package psycopg2 > > C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt > > ion: 'summary' > > warnings.warn(msg) > > > > Installing collected packages: psycopg2 > > Running setup.py install for psycopg2 > > C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution opt > > ion: 'summary' > > warnings.warn(msg) > > > > Successfully installed psycopg2 > > Cleaning up... > > Exception: > > Traceback (most recent call last): > > File "C:\Python34\lib\shutil.py", line 370, in _rmtree_unsafe > > os.unlink(fullname) > > PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Semantic\\AppData\\L > > ocal\\Temp\\pip_build_Semantic\\psycopg2\\.git\\objects\\pack\\pack-be4d3da4a06b > > 4c9ec4c06040dbf6685eeccca068.idx' > > > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > File "C:\Python34\lib\site-packages\pip\basecommand.py", line 122, in main > > status = self.run(options, args) > > File "C:\Python34\lib\site-packages\pip\commands\install.py", line 302, in run > > > > requirement_set.cleanup_files(bundle=self.bundle) > > File "C:\Python34\lib\site-packages\pip\req.py", line 1333, in cleanup_files > > rmtree(dir) > > File "C:\Python34\lib\site-packages\pip\util.py", line 43, in rmtree > > onerror=rmtree_errorhandler) > > File "C:\Python34\lib\shutil.py", line 477, in rmtree > > return _rmtree_unsafe(path, onerror) > > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > > _rmtree_unsafe(fullname, onerror) > > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > > _rmtree_unsafe(fullname, onerror) > > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > > _rmtree_unsafe(fullname, onerror) > > File "C:\Python34\lib\shutil.py", line 367, in _rmtree_unsafe > > _rmtree_unsafe(fullname, onerror) > > File "C:\Python34\lib\shutil.py", line 372, in _rmtree_unsafe > > onerror(os.unlink, fullname, sys.exc_info()) > > File "C:\Python34\lib\site-packages\pip\util.py", line 53, in rmtree_errorhand > > ler > > (exctype is PermissionError and value.args[3] == 5) #python3.3 > > IndexError: tuple index out of range > > > > The above clearly shows "Successfully installed psycopg2" and that it's > a permission error on cleanup that's gone wrong, so what is there to > report on the bug tracker? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence 1. I did not mean to confuse you by reference to the bug tracker. My log in difficulties are not related to this issue. 2. The other reference to the bug tracker was to indicate that I don't think this is the same error as mentioned there. 3. Despite the report of success, I do not have psycopg2. Since that failure was followed by the permission errors, I assume they are related. This is why I posted, to get help with this problem. I look forward to any assistance you or anyone else can render on this issue. Thanks. 4. Going back to bug #14252, it is structurally very similar. I forget the program at issue there, but at first it reported success and that was followed by win error 5, and in fact the program had not installed correctly. The difference is that #14252 involved subprocess.py and my traceback does not mention subprocess.py at all. From malik.a.rumi at gmail.com Thu Feb 26 16:56:08 2015 From: malik.a.rumi at gmail.com (Malik Rumi) Date: Thu, 26 Feb 2015 13:56:08 -0800 (PST) Subject: Is anyone else unable to log into the bug tracker? In-Reply-To: References: <54b0850a$0$13002$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thursday, February 26, 2015 at 10:49:19 AM UTC-6, Skip Montanaro wrote: > I have not had problems, but I use the Google login (Open ID, I presume) option. > > > Skip Ok, I got it. In short, capitalization (or not) matters. Thanks to all. From rosuav at gmail.com Thu Feb 26 17:13:57 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 09:13:57 +1100 Subject: Newbie question about text encoding In-Reply-To: <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> Message-ID: On Fri, Feb 27, 2015 at 4:59 AM, Rustom Mody wrote: > On Thursday, February 26, 2015 at 10:33:44 PM UTC+5:30, Terry Reedy wrote: >> I think that this part of your post is more 'unprofessional' than the >> character blocks. It is very jarring and seems contrary to your main point. > > Ok I need a word for > 1. I have no need for this > 2. 99.9% of the (living) on this planet also have no need for this So what, seven million people need it? Sounds pretty useful to me. And your figure is an exaggeration; a lot more people than that use emoji/emoticons. >> > Also, how does assigning meanings to codepoints "waste storage"? As >> > soon as Unicode 2.0 hit and 16-bit code units stopped being >> > sufficient, everyone needed to allocate storage - either 32 bits per >> > character, or some other system - and the fact that some codepoints >> > were unassigned had absolutely no impact on that. This is decidedly >> > NOT unprofessional, and it's not wasteful either. >> >> I agree. > > I clearly am more enthusiastic than knowledgeable about unicode. > But I know my basic CS well enough (as I am sure you and Chris also do) > > So I dont get how 4 bytes is not more expensive than 2. > Yeah I know you can squeeze a unicode char into 3 bytes or even 21 bits > You could use a clever representation like UTF-8 or FSR. > But I dont see how you can get out of this that full-unicode costs more than > exclusive BMP. Sure, UCS-2 is cheaper than the current Unicode spec. But Unicode 2.0 was when that changed, and the change was because 65536 characters clearly wouldn't be enough - and that was due to the number of characters needed for other things than those you're complaining about. Every spec since then has not changed anything that affects storage. There are still, today, quite a lot of unallocated blocks of characters (we're really using only about two planes' worth so far, maybe three), but even if Unicode specified just two planes of 64K characters each, you wouldn't be able to save much on transmission (UTF-8 is already flexible and uses only what you need; if a future Unicode spec allows 64K planes, UTF-8 transmission will cost exactly the same for all existing characters), and on an eight-bit-byte system, the very best you'll be able to do is three bytes - which you can do today, too; you already know 21 bits will do. So since the BMP was proven insufficient (back in 1996), no subsequent changes have had any costs in storage. > Still if I were to expand on the criticisms here are some examples: > > Math-Greek: Consider the math-alpha block > http://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode#Mathematical_Alphanumeric_Symbols_block > > Now imagine a beginning student not getting the difference between font, glyph, > character. To me this block represents this same error cast into concrete and > dignified by the (supposed) authority of the unicode consortium. > > There are probably dozens of other such stupidities like distinguishing kelvin ? from latin K as if that is the business of the unicode consortium A lot of these kinds of characters come from a need to unambiguously transliterate text stored in other encodings. I don't personally profess to understand the reasoning behind the various indistinguishable characters, but I'm aware that there are a lot of tricky questions to be decided; and if once the Consortium decides to allocate a character, that character must remain forever allocated. > My real reservations about unicode come from their work in areas that I happen to know something about > > Music: To put music simply as a few mostly-meaningless 'dingbats' like ? ? ? is perhaps ok > However all this stuff http://xahlee.info/comp/unicode_music_symbols.html > makes no sense (to me) given that music (ie standard western music written in staff notation) is inherently 2 dimensional -- multi-voiced, multi-staff, chordal The placement on the page is up to the display library. You can produce a PDF that places the note symbols at their correct positions, and requires no images to render sheet music. > Sanskrit/Devanagari: > Consists of bogus letters that dont exist in devanagari > The letter ? (0904) is found here http://unicode.org/charts/PDF/U0900.pdf > But not here http://en.wikipedia.org/wiki/Devanagari#Vowels > So I call it bogus-devanagari > > Contrariwise an important letter in vedic pronunciation the double-udatta is missing > http://list.indology.info/pipermail/indology_list.indology.info/2000-April/021070.html > > All of which adds up to the impression that the unicode consortium occasionally fails to do due diligence Which proves that they're not perfect. Don't forget, they can always add more characters later. ChrisA From jason.swails at gmail.com Thu Feb 26 17:28:56 2015 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 26 Feb 2015 17:28:56 -0500 Subject: Parallelization of Python on GPU? In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> <1424972883.30927.138.camel@gmail.com> Message-ID: On Thu, Feb 26, 2015 at 4:10 PM, Sturla Molden wrote: > On 26/02/15 18:48, Jason Swails wrote: > >> On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote: >> >>> GPU computing is great if you have the following: >>> >>> 1. Your data structures are arrays floating point numbers. >>> >> >> It actually works equally great, if not better, for integers. >> > > Right, but not complicated data structures with a lot of references or > pointers. It requires data are laid out in regular arrays, and then it acts > on these arrays in a data-parallel manner. It is designed to process > vertices in parallel for computer graphics, and that is a limitation which > is always there. It is not a CPU with 1024 cores. It is a "floating point > monster" which can process 1024 vectors in parallel. You write a tiny > kernel in a C-like language (CUDA, OpenCL) to process one vector, and then > it will apply the kernel to all the vectors in an array of vectors. It is > very comparable to how GLSL and Direct3D vertex and fragment shaders work. > (The reason for which should be obvious.) The GPU is actually great for a > lot of things in science, but it is not a CPU. The biggest mistake in the > GPGPU hype is the idea that the GPU will behave like a CPU with many cores. Very well summarized. At least in my field, though, it is well-known that GPUs are not 'uber-fast CPUs'. Algorithms have been redesigned, programs rewritten to take advantage of their architecture. It has been a *massive* investment of time and resources, but (unlike the Xeon Phi coprocessor [1]) has reaped most of its promised rewards. ?--Jason [1] I couldn't resist the jab. At several times the cost of the top of the line NVidia gaming card, the GPU is about 15-20x faster... -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve+comp.lang.python at pearwood.info Thu Feb 26 18:09:41 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 27 Feb 2015 10:09:41 +1100 Subject: Newbie question about text encoding References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> Message-ID: <54efa7b6$0$12994$c3e8da3$5496439d@news.astraweb.com> Chris Angelico wrote: > Unicode > isn't about taking everyone's separate character sets and numbering > them all so we can reference characters from anywhere; if you wanted > that, you'd be much better off with something that lets you specify a > code page in 16 bits and a character in 8, which is roughly the same > size as Unicode anyway. Well, except for the approximately 25% of people in the world whose native language has more than 256 characters. It sounds like you are referring to some sort of "shift code" system. Some legacy East Asian encodings use a similar scheme, and depending on how they are implemented they have great disadvantages. For example, Shift-JIS suffers from a number of weaknesses including that a single byte corrupted in transmission can cause large swaths of the following text to be corrupted. With Unicode, a single corrupted byte can only corrupt a single code point. -- Steven From rosuav at gmail.com Thu Feb 26 18:23:29 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 10:23:29 +1100 Subject: Newbie question about text encoding In-Reply-To: <54efa7b6$0$12994$c3e8da3$5496439d@news.astraweb.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <54efa7b6$0$12994$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Feb 27, 2015 at 10:09 AM, Steven D'Aprano wrote: > Chris Angelico wrote: > >> Unicode >> isn't about taking everyone's separate character sets and numbering >> them all so we can reference characters from anywhere; if you wanted >> that, you'd be much better off with something that lets you specify a >> code page in 16 bits and a character in 8, which is roughly the same >> size as Unicode anyway. > > Well, except for the approximately 25% of people in the world whose native > language has more than 256 characters. You could always allocate multiple code pages to one language. But since I'm not advocating this system, I'm only guessing at solutions to its problems. > It sounds like you are referring to some sort of "shift code" system. Some > legacy East Asian encodings use a similar scheme, and depending on how they > are implemented they have great disadvantages. For example, Shift-JIS > suffers from a number of weaknesses including that a single byte corrupted > in transmission can cause large swaths of the following text to be > corrupted. With Unicode, a single corrupted byte can only corrupt a single > code point. That's exactly what I was hinting at. There are plenty of systems like that, and they are badly flawed compared to a simple universal system for a number of reasons. One is the corruption issue you mention; another is that a simple memory-based text search becomes utterly useless (to locate text in a document, you'd need to do a whole lot of stateful parsing - not to mention the difficulties of doing "similar-to" searches across languages); concatenation of text also becomes a stateful operation, and so do all sorts of other simple manipulations. Unicode may demand a bit more storage in certain circumstances (where an eight-bit encoding might have handled your entire document), but it's so much easier for the general case. ChrisA From fabiofz at gmail.com Thu Feb 26 19:04:24 2015 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Thu, 26 Feb 2015 21:04:24 -0300 Subject: Design thought for callbacks In-Reply-To: <575FE33B-D1AD-4391-B816-7440D5849C41@gmail.com> References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com> <575FE33B-D1AD-4391-B816-7440D5849C41@gmail.com> Message-ID: On Wed, Feb 25, 2015 at 9:46 AM, Cem Karan wrote: > > On Feb 24, 2015, at 8:23 AM, Fabio Zadrozny wrote: > > > Hi Cem, > > > > I didn't read the whole long thread, but I thought I'd point you to what > I'm using in PyVmMonitor (http://www.pyvmmonitor.com/) -- which may > already cover your use-case. > > > > Take a look at the callback.py at > https://github.com/fabioz/pyvmmonitor-core/blob/master/pyvmmonitor_core/callback.py > > > > And its related test (where you can see how to use it): > https://github.com/fabioz/pyvmmonitor-core/blob/master/_pyvmmonitor_core_tests/test_callback.py > (note that it falls back to a strong reference on simple functions -- i.e.: > usually top-level methods or methods created inside a scope -- but > otherwise uses weak references). > > That looks like a better version of what I was thinking about originally. > However, various people on the list have convinced me to stick with strong > references everywhere. I'm working out a possible API right now, once I > have some code that I can use to illustrate what I'm thinking to everyone, > I'll post it to the list. > > Thank you for showing me your code though, it is clever! > > Thanks, > Cem Karan ?Hi Cem, Well, I decided to elaborate a bit on the use-case I have and how I use it (on a higher level): http://pydev.blogspot.com.br/2015/02/design-for-client-side-applications-in.html So, you can see if it may be worth for you or not (I agree that sometimes you should keep strong references, but for my use-cases, weak references usually work better -- with the only exception being closures, which is handled different anyways but with the gotcha of having to manually unregister it). Best Regards, Fabio? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Feb 26 19:12:53 2015 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 27 Feb 2015 11:12:53 +1100 Subject: ANN: Wing IDE 5.1.2 released In-Reply-To: References: Message-ID: On Fri, Feb 27, 2015 at 6:42 AM, William Ray Wing wrote: > PS: I?ve found that the Wing e-mail support is VERY responsive. No relation, just a happy user. You should totally get involved with the project. With your name, everyone would think you started it! ChrisA From steve+comp.lang.python at pearwood.info Thu Feb 26 20:05:10 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 27 Feb 2015 12:05:10 +1100 Subject: Newbie question about text encoding References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> Message-ID: <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> Rustom Mody wrote: > Emoticons (or is it emoji) seems to have some (regional?) takeup?? Dunno? > In any case I'd like to stay clear of political(izable) questions Emoji is the term used in Japan, gradually spreading to the rest of the word. Emoticons, I believe, should be restricted to the practice of using ASCII-only digraphs and trigraphs such as :-) (colon, hyphen, right-parens) to indicate "smileys". I believe that emoji will eventually lead to Unicode's victory. People will want smileys and piles of poo on their mobile phones, and from there it will gradually spread to everywhere. All they need to do to make victory inevitable is add cartoon genitals... >> I think that this part of your post is more 'unprofessional' than the >> character blocks. It is very jarring and seems contrary to your main >> point. > > Ok I need a word for > 1. I have no need for this > 2. 99.9% of the (living) on this planet also have no need for this 0.1% of the living is seven million people. I'll tell you what, you tell me which seven million people should be relegated to second-class status, and I'll tell them where you live. :-) [...] > I clearly am more enthusiastic than knowledgeable about unicode. > But I know my basic CS well enough (as I am sure you and Chris also do) > > So I dont get how 4 bytes is not more expensive than 2. Obviously it is. But it's only twice as expensive, and in computer science terms that counts as "close enough". It's quite common for data structures to "waste" space by using "no more than twice as much space as needed", e.g. Python dicts and lists. The whole Unicode range U+0000 to U+10FFFF needs only 21 bits, which fits into three bytes. Nevertheless, there's no three-byte UTF encoding, because on modern hardware it is more efficient to "waste" an entire extra byte per code point and deal with an even multiple of bytes. > Yeah I know you can squeeze a unicode char into 3 bytes or even 21 bits > You could use a clever representation like UTF-8 or FSR. > But I dont see how you can get out of this that full-unicode costs more > than exclusive BMP. Are you missing a word there? Costs "no more" perhaps? > eg consider the case of 32 vs 64 bit executables. > The 64 bit executable is generally larger than the 32 bit one > Now consider the case of a machine that has say 2GB RAM and a 64-bit > processor. You could -- I think -- make a reasonable case that all those > all-zero hi-address-words are 'waste'. Sure. The whole point of 64-bit processors is to enable the use of more than 2GB of RAM. One might as well say that using 32-bit processors is wasteful if you only have 64K of memory. Yes it is, but the only things which use 16-bit or 8-bit processors these days are embedded devices. [...] > Math-Greek: Consider the math-alpha block > http://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode#Mathematical_Alphanumeric_Symbols_block > > Now imagine a beginning student not getting the difference between font, > glyph, > character. To me this block represents this same error cast into concrete > and dignified by the (supposed) authority of the unicode consortium. Not being privy to the internal deliberations of the Consortium, it is sometimes difficult to tell why two symbols are sometimes declared to be mere different glyphs for the same character, and other times declared to be worthy of being separate characters. E.g. I think we should all agree that the English "A" and the French "A" shouldn't count as separate characters, although the Greek "?" and Russian "?" do. In the case of the maths symbols, it isn't obvious to me what the deciding factors were. I know that one of the considerations they use is to consider whether or not users of the symbols have a tradition of treating the symbols as mere different glyphs, i.e. stylistic variations. In this case, I'm pretty sure that mathematicians would *not* consider: U+2115 DOUBLE-STRUCK CAPITAL N "?" U+004E LATIN CAPITAL LETTER N "N" as mere stylistic variations. If you defined a matrix called ?, you would probably be told off for using the wrong symbol, not for using the wrong formatting. On the other hand, I'm not so sure about U+210E PLANCK CONSTANT "?" versus a mere lowercase h (possibly in italic). > There are probably dozens of other such stupidities like distinguishing > kelvin ? from latin K as if that is the business of the unicode consortium But it *is* the business of the Unicode consortium. They have at least two important aims: - to be able to represent every possible human-language character; - to allow lossless round-trip conversion to all existing legacy encodings (for the subset of Unicode handled by that encoding). The second reason is why Unicode includes code points for degree-Celsius and degree-Fahrenheit, rather than just using ?C and ?F like sane people. Because some idiot^W code-page designer back in the 1980s or 90s decided to add single character ? and ?. So now Unicode has to be able to round-trip (say) "?C?" without loss. I imagine that the same applies to U+212A KELVIN SIGN ?. > My real reservations about unicode come from their work in areas that I > happen to know something about > > Music: To put music simply as a few mostly-meaningless 'dingbats' like ? ? > ? is perhaps ok However all this stuff > http://xahlee.info/comp/unicode_music_symbols.html > makes no sense (to me) given that music (ie standard western music written > in staff notation) is inherently 2 dimensional -- multi-voiced, > multi-staff, chordal (1) Text can also be two dimensional. (2) Where you put the symbol on the page is a separate question from whether or not the symbol exists. > Consists of bogus letters that dont exist in devanagari > The letter ? (0904) is found here http://unicode.org/charts/PDF/U0900.pdf > But not here http://en.wikipedia.org/wiki/Devanagari#Vowels > So I call it bogus-devanagari Hmm, well I love Wikipedia as much as the next guy, but I think that even Jimmy Wales would suggest that Wikipedia is not a primary source for what counts as Devanagari vowels. What makes you think that Wikipedia is right and Unicode is wrong? That's not to say that Unicode hasn't made some mistakes. There are a few deprecated code points, or code points that have been given the wrong name. Oops. Mistakes happen. > Contrariwise an important letter in vedic pronunciation the double-udatta > is missing > http://list.indology.info/pipermail/indology_list.indology.info/2000-April/021070.html I quote: I do not see any need for a "double udaatta". Perhaps "double ANudaatta" is meant here? I don't know Sanskrit, but if somebody suggested that Unicode doesn't support English because the important letter "double-oh" (as in "moon", "spoon", "croon" etc.) was missing, I wouldn't be terribly impressed. We have a "double-u" letter, why not "double-oh"? Another quote: I should strongly recommend not to hurry with a standardization proposal until the text collection of Vedic texts has been finished In other words, even the experts in Vedic texts don't yet know all the characters which they may or may not need. -- Steven From davea at davea.name Thu Feb 26 20:57:42 2015 From: davea at davea.name (Dave Angel) Date: Thu, 26 Feb 2015 20:57:42 -0500 Subject: Newbie question about text encoding In-Reply-To: <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54EFCF16.80908@davea.name> On 02/26/2015 08:05 PM, Steven D'Aprano wrote: > Rustom Mody wrote: > > >> eg consider the case of 32 vs 64 bit executables. >> The 64 bit executable is generally larger than the 32 bit one >> Now consider the case of a machine that has say 2GB RAM and a 64-bit >> processor. You could -- I think -- make a reasonable case that all those >> all-zero hi-address-words are 'waste'. > > Sure. The whole point of 64-bit processors is to enable the use of more than > 2GB of RAM. One might as well say that using 32-bit processors is wasteful > if you only have 64K of memory. Yes it is, but the only things which use > 16-bit or 8-bit processors these days are embedded devices. But the 2gig means electrical address lines out of the CPU are wasted, not address space. A 64 bit processor and 64bit OS means you can have more than 4gig in a process space, even if over half of it has to be in the swap file. Linear versus physical makes a big difference. (Although I believe Seymour Cray was quoted as saying that virtual memory is a crock, because "you can't fake what you ain't got.") -- DaveA From memilanuk at gmail.com Thu Feb 26 22:53:34 2015 From: memilanuk at gmail.com (memilanuk) Date: Thu, 26 Feb 2015 19:53:34 -0800 Subject: Picking apart a text line Message-ID: So... okay. I've got a bunch of PDFs of tournament reports that I want to sift thru for information. Ended up using 'pdftotext -layout file.pdf file.txt' to extract the text from the PDF. Still have a few little glitches to iron out there, but I'm getting decent enough results for the moment to move on. I've got my script to where it opens the file, ignores the header lines at the top, then goes through the rest of the file line by line, skipping lines if they don't match (don't need the separator lines) and adding them to a list if they do (and stripping whitespace off the right side along the way). So far, so good. # rstatPDF2csv.py import sys import re def convert(file): lines = [] data = open(file) # Skip first n lines of headers for i in range(9): data.__next__() # Read remaining lines one at a time for line in data: # If the line begins with a capital letter... if re.match(r'^[A-Z]', line): # Strip any trailing whitespace and then add to the list lines.append(line.rstrip()) return lines if __name__ == '__main__': print(convert(sys.argv[1])) What I'm ending up with is a list full of strings that look something like this: ['JOHN DOE C T HM 445-20*MW* 199-11*MW* 194-5 1HM 393-16*MW* 198-9 1HM 198-11*MW* 396-20*MW* 789-36*MW* 1234-56 *MW*', Basically... a certain number of characters allotted for competitor name, then four or five 1-2 char columns for things like classification, age group, special categories, etc., then a score ('445-20'), then up to 4 char for award (if any), then another score, another award, etc. etc. etc. Right now (in the PDF) the scores are batched by one criterion, then sorted within those groups. Makes life easier for the person giving out awards at the end of the tournament, not so much for someone trying to see how their individual score ranks against the whole field, not just their group or sub-group. I want to be able to pull all the scores out and then re-sort based on score - mainly the final aggregate score, but potentially also on stage or daily scores. Eventually I'd like to be able to calculate standardized z-scores so as to be able to compare scores from one event/location against another. So back to the lines of text I have stored as strings in a list. I think I want to convert that to a list of lists, i.e. split each line up, store that info in another list and ditch the whitespace. Or would I be better off using dicts? Originally I was thinking of how to process each line and split it them up based on what information was where - some sort of nested for/if mess. Now I'm starting to think that the lines of text are pretty uniform in structure i.e. the same field is always in the same location, and that list slicing might be the way to go, if a bit tedious to set up initially...? Any thoughts or suggestions from people who've gone down this particular path would be greatly appreciated. I think I have a general idea/direction, but I'm open to other ideas if the path I'm on is just blatantly wrong. Thanks, Monte -- Shiny! Let's be bad guys. Reach me @ memilanuk (at) gmail dot com From no.email at nospam.invalid Fri Feb 27 00:55:38 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 26 Feb 2015 21:55:38 -0800 Subject: Are threads bad? - was: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <87lhjnang1.fsf@jester.gateway.pace.com> Message-ID: <87h9u799ut.fsf@jester.gateway.pace.com> Ryan Stuart writes: > My point is malloc, something further up (down?) the stack, is making > modifications to shared state when threads are involved. Modifying > shared state makes it infinitely more difficult to reason about the > correctness of your software. If you're saying the libc malloc might have bugs that affect multithreaded apps but not single threaded ones, then sure, but the Linux kernel might also have such bugs and it's inherently multithreaded, so there's no escape. Even if your app is threaded you're still susceptible to threading bugs in the kernel. If malloc works properly then it's thread-safe and you can use it without worrying about how your app's state interacts with malloc's internal state. > We clearly got completely different things from the article. My > interpretation was that it was making *the exact opposite* point to > what you stated mainly because non-threading approaches don't share > state. It gave the example of asyncio, which is non-threaded but (according to the article) was susceptible to shared state bugs because you could accidentally insert yield points in critical sections, by doing things like logging. > It states that quite clearly. For example "it is ? literally ? > exponentially more difficult to reason about a routine that may be > executed from an arbitrary number of threads concurrently". I didn't understand what it was getting at with that n**n claim. Of course arbitrary code (even single threaded) is incalculably difficult to reason about (halting problem, Rice's theorem). But we're talking about code following a particular set of conventions, not arbitrary code. The conventions are supposed to facilitate reasoning and verification. Again there's tons of solid theory in the OS literature about this stuff. > by default Haskell looks to use lightweight threads where only 1 > thread can be executing at a time [1]... That doesn't seem to be > shared state multithreading, which is what the article is referring to. Haskell uses lightweight, shared state threads with synchronization primitives that do the usual things (the API is somewhat different than Posix threads though). You have to use the +RTS command line option to run on multiple cores: I don't know why the default is to stay on a single core. There might be a performance hit if you use the multicore runtime with a single-threaded program, or something like that. There is a book about Haskell concurrency and parallelism that I've been wanting to read (full text online): http://chimera.labs.oreilly.com/books/1230000000929/index.html > 2) it has a weird story about the brass cockroach, that basically > signified that they didn't have a robust enough testing system to > be able to reproduce the bug. > > The point was that it wasn't feasible to have a robust testing suite > because, you guessed it, No really, they observed this bug happening repeatedly under what sounded like fairly light load with real users. So a stress testing framework should have been able to reproduce it. Do you really think it's impossible to debug this kind of problem? OS developers do it all the time. There is no getting around it. > This is probably correct. Is there any STM implementations out that > that don't significantly compromise performance? STM is fast as long as there's not much contention for shared data between threads. In the "account balance" example that should almost always be the case. The slowdown is when multiple threads are fighting over the same data and transactions keep having to be rolled back and restarted. > multiprocessing module looks pretty nice and I should try it > It's 1 real advantage is that it side-steps the GIL. So, if you need > to utilise multiple cores for CPU bound tasks, then it might well be > the only option. It's 1 real advantage compared to what? I thought you were saying it avoids shared data hazards of threads. The 4 alternatives in that article were threads, multiprocessing, old-fashioned async (callback hell), and asyncio (still contorted and relies on Python 3 coroutines). If you eliminate threads because of data sharing and asyncio because you need Python 2 compatibility, you're left with multiprocessing if you want to avoid the control inversion of callback style. It's true though, this started out about the GIL in PyPy (was Laura going to post about that?) so using multicores is indeed maybe relevant. From steve+comp.lang.python at pearwood.info Fri Feb 27 00:58:30 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 27 Feb 2015 16:58:30 +1100 Subject: Newbie question about text encoding References: <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> Dave Angel wrote: > (Although I believe Seymour Cray was quoted as saying that virtual > memory is a crock, because "you can't fake what you ain't got.") If I recall correctly, disk access is about 10000 times slower than RAM, so virtual memory is *at least* that much slower than real memory. -- Steven From frank at chagford.com Fri Feb 27 01:04:39 2015 From: frank at chagford.com (Frank Millman) Date: Fri, 27 Feb 2015 08:04:39 +0200 Subject: Python in The Economist Message-ID: Hi all >From a recent article in The Economist - "A recovering economy in America and an explosion of entrepreneurial activity are driving up demand for tech talent. [...] Bidding battles are breaking out, with salaries and bonuses rising fast for experts in popular computer languages such as Python and Ruby on Rails." The author seems to have obtained his information from "a recent dinner party in Silicon Valley", so it may not be very representative. But to be mentioned in such a high-profile newspaper with its international readership can only be good for Python. Here is a link to the full article - http://www.economist.com/news/business/21644150-battle-software-talent-other-industries-can-learn-silicon-valley-how-bag Frank Millman From davea at davea.name Fri Feb 27 02:30:46 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 02:30:46 -0500 Subject: Newbie question about text encoding In-Reply-To: <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> References: <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54F01D26.1080705@davea.name> On 02/27/2015 12:58 AM, Steven D'Aprano wrote: > Dave Angel wrote: > >> (Although I believe Seymour Cray was quoted as saying that virtual >> memory is a crock, because "you can't fake what you ain't got.") > > If I recall correctly, disk access is about 10000 times slower than RAM, so > virtual memory is *at least* that much slower than real memory. > It's so much more complicated than that, that I hardly know where to start. I'll describe a generic processor/OS/memory/disk architecture; there will be huge differences between processor models even from a single manufacturer. First, as soon as you add swapping logic to your processor/memory-system, you theoretically slow it down. And in the days of that quote, Cray's memory was maybe 50 times as fast as the memory used by us mortals. So adding swapping logic would have slowed it down quite substantially, even when it was not swapping. But that logic is inside the CPU chip these days, and presumably thoroughly optimized. Next, statistically, a program uses a small subset of its total program & data space in its working set, and the working set should reside in real memory. But when the program greatly increases that working set, and it approaches the amount of physical memory, then swapping becomes more frenzied, and we say the program is thrashing. Simple example, try sorting an array that's about the size of available physical memory. Next, even physical memory is divided into a few levels of caching, some on-chip and some off. And the caching is done in what I call strips, where accessing just one byte causes the whole strip to be loaded from non-cached memory. I forget the current size for that, but it's maybe 64 to 256 bytes or so. If there are multiple processors (not multicore, but actual separate processors), then each one has such internal caches, and any writes on one processor may have to trigger flushes of all the other processors that happen to have the same strip loaded. The processor not only prefetches the next few instructions, but decodes and tentatively executes them, subject to being discarded if a conditional branch doesn't go the way the processor predicted. So some instructions execute in zero time, some of the time. Every address of instruction fetch, or of data fetch or store, goes through a couple of layers of translation. Segment register plus offset gives linear address. Lookup those in tables to get physical address, and if table happens not to be in on-chip cache, swap it in. If physical address isn't valid, a processor exception causes the OS to potentially swap something out, and something else in. Once we're paging from the swapfile, the size of the read is perhaps 4k. And that read is regardless of whether we're only going to use one byte or all of it. The ratio between an access which was in the L1 cache and one which required a page to be swapped in from disk? Much bigger than your 10,000 figure. But hopefully it doesn't happen a big percentage of the time. Many, many other variables, like the fact that RAM chips are not directly addressable by bytes, but instead count on rows and columns. So if you access many bytes in the same row, it can be much quicker than random access. So simple access time specifications don't mean as much as it would seem; the controller has to balance the RAM spec with the various cache requirements. -- DaveA From davea at davea.name Fri Feb 27 02:39:58 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 02:39:58 -0500 Subject: Picking apart a text line In-Reply-To: References: Message-ID: <54F01F4E.3030708@davea.name> On 02/26/2015 10:53 PM, memilanuk wrote: > So... okay. I've got a bunch of PDFs of tournament reports that I want > to sift thru for information. Ended up using 'pdftotext -layout > file.pdf file.txt' to extract the text from the PDF. Still have a few > little glitches to iron out there, but I'm getting decent enough results > for the moment to move on. > > I've got my script to where it opens the file, ignores the header lines > at the top, then goes through the rest of the file line by line, > skipping lines if they don't match (don't need the separator lines) and > adding them to a list if they do (and stripping whitespace off the right > side along the way). So far, so good. > > # rstatPDF2csv.py > > import sys > import re > > > def convert(file): > lines = [] > data = open(file) > > # Skip first n lines of headers > for i in range(9): > data.__next__() > > # Read remaining lines one at a time > for line in data: > > # If the line begins with a capital letter... > if re.match(r'^[A-Z]', line): > > # Strip any trailing whitespace and then add to the list > lines.append(line.rstrip()) > > return lines > > if __name__ == '__main__': > print(convert(sys.argv[1])) > > > > What I'm ending up with is a list full of strings that look something > like this: > > ['JOHN DOE C T HM 445-20*MW* 199-11*MW* 194-5 > 1HM 393-16*MW* 198-9 1HM 198-11*MW* 396-20*MW* > 789-36*MW* 1234-56 *MW*', > > Basically... a certain number of characters allotted for competitor > name, then four or five 1-2 char columns for things like classification, > age group, special categories, etc., then a score ('445-20'), then up to > 4 char for award (if any), then another score, another award, etc. etc. > etc. > > Right now (in the PDF) the scores are batched by one criterion, then > sorted within those groups. Makes life easier for the person giving out > awards at the end of the tournament, not so much for someone trying to > see how their individual score ranks against the whole field, not just > their group or sub-group. I want to be able to pull all the scores out > and then re-sort based on score - mainly the final aggregate score, but > potentially also on stage or daily scores. Eventually I'd like to be > able to calculate standardized z-scores so as to be able to compare > scores from one event/location against another. > > So back to the lines of text I have stored as strings in a list. I > think I want to convert that to a list of lists, i.e. split each line > up, store that info in another list and ditch the whitespace. Or would > I be better off using dicts? Originally I was thinking of how to > process each line and split it them up based on what information was > where - some sort of nested for/if mess. Now I'm starting to think that > the lines of text are pretty uniform in structure i.e. the same field is > always in the same location, and that list slicing might be the way to > go, if a bit tedious to set up initially...? > > Any thoughts or suggestions from people who've gone down this particular > path would be greatly appreciated. I think I have a general > idea/direction, but I'm open to other ideas if the path I'm on is just > blatantly wrong. > Maintaining a list of lists is a big pain. If the data is truly very uniform, you might want to do it, but I'd find it much more reasonable to have names for the fields of each line. You can either do that with a named-tuple, or with instances of a custom class of your own. See https://docs.python.org/3.4/library/collections.html#namedtuple-factory-function-for-tuples-with-named-fields You read a line, do some sanity checking on it, and construct an object. Go to the next line, do the same, another object. Those objects are stored in a list. Everything else accesses the fields of the object something like: for row in mylist: print( row.name, row.classification, row.age) if row.name == "Doe": ... -- DaveA From breamoreboy at yahoo.co.uk Fri Feb 27 03:02:52 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 27 Feb 2015 08:02:52 +0000 Subject: Python Worst Practices In-Reply-To: References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> Message-ID: On 27/02/2015 01:31, Dennis Lee Bieber wrote: > On Thu, 26 Feb 2015 20:10:28 +0000, Simon Ward > declaimed the following: > >> >> 0 = success and non-zero = failure is the meme established, rather than 0 = true, non-zero = false. >> >> It's not just used by UNIX, and is not necessarily defined by the shell either (bash was mentioned elsewhere in the thread). There is probably a system that pre-dates UNIX that I uses/used this too, but I don't know. >> > > I miss VMS... > > Odd => true, even => false > Odd -> success, +even -> warning, -even -> error (odd may have been > success vs info> > Me to, Very Much Safer :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From wxjmfauth at gmail.com Fri Feb 27 04:06:57 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Fri, 27 Feb 2015 01:06:57 -0800 (PST) Subject: Newbie question about text encoding In-Reply-To: <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> References: <54EC5FA4.6070703@davea.name> <201502241455.t1OEtffT016452@fido.openend.se> <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> Message-ID: Le vendredi 27 f?vrier 2015 02:05:38 UTC+1, Steven D'Aprano a ?crit?: > > E.g. I think we should all agree that the English "A" and the French "A" > shouldn't count as separate characters, although the Greek "?" and > Russian "?" do. > Yes. Simple and logical explaination. Unicode does not handle languages per se, it encodes scripts for languages. From mal at egenix.com Fri Feb 27 05:08:05 2015 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 27 Feb 2015 11:08:05 +0100 Subject: mx as dependency tox throws GCC error In-Reply-To: <1569855376.2617820.1424875408718.JavaMail.yahoo@mail.yahoo.com> References: <1569855376.2617820.1424875408718.JavaMail.yahoo@mail.yahoo.com> Message-ID: <54F04205.4060909@egenix.com> On 25.02.2015 15:43, Albert-Jan Roskam wrote: > Hi, > > If I pip install the mx package with "pip install egenix-mx-base", it works. > If I put that same pip install command under 'install_command' in my tox.ini it also works (see below) > > However, if I specify the dependency under 'deps', I get an error. Any idea why? I read that 'install_command' is > still experimental and subject to change, so I would prefer to specify the package under 'deps'. > The error occurred when I called tox using 'tox -e py27', so it is not pypy-related. > > > [tox] > envlist = pypy,py27,py33,py34,docs > skipsdist = true > > [testenv] > deps = > setuptools > nose > gocept.testing > numpy > cython > # egenix-mx-base # --> mx tox gcc: error: mx/DateTime/mxDateTime/mxDateTime.c: No such file or directory > install_command = > pip install egenix-mx-base {packages} Could you provide some more context or debug output ? The .c file is definitely part of the egenix-mx-base package, but it's possible that tox is using a different way of installing the package than pip itself does when using it under "deps". If there's a way to work around the problem, we'll fix this in our mxSetup.py. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 27 2015) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From bintadoma20 at yahoo.com Fri Feb 27 05:18:24 2015 From: bintadoma20 at yahoo.com (babyG) Date: Fri, 27 Feb 2015 04:18:24 -0600 Subject: strip bug? References: <3a54074b-2517-472b-9fcc-50348933a959@googlegroups.com> Message-ID: Hello how are you doing please From d.a.a.deepal at gmail.com Fri Feb 27 06:03:32 2015 From: d.a.a.deepal at gmail.com (Amila Deepal) Date: Fri, 27 Feb 2015 03:03:32 -0800 (PST) Subject: please help to run pylearn2 Message-ID: <73f31f8f-0e3c-4cf3-abc2-1785916ade13@googlegroups.com> I try to run pylearn2 tutorial: Softmax regression using my notebook. but i run from pylearn2.config import yaml_parse train = yaml_parse.load(train) train.main_loop() this code in my notebook i got Error.How to solve this Please help --------------------------------------------------------------------------- ImportError Traceback (most recent call last) in () 1 from pylearn2.config import yaml_parse ----> 2 train = yaml_parse.load(train) 3 train.main_loop() /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in load(stream, environ, instantiate, **kwargs) 209 string = stream.read() 210 --> 211 proxy_graph = yaml.load(string, **kwargs) 212 if instantiate: 213 return _instantiate(proxy_graph) /usr/lib/python2.7/dist-packages/yaml/__init__.pyc in load(stream, Loader) 69 loader = Loader(stream) 70 try: ---> 71 return loader.get_single_data() 72 finally: 73 loader.dispose() /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in get_single_data(self) 37 node = self.get_single_node() 38 if node is not None: ---> 39 return self.construct_document(node) 40 return None 41 /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in construct_document(self, node) 41 42 def construct_document(self, node): ---> 43 data = self.construct_object(node) 44 while self.state_generators: 45 state_generators = self.state_generators /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in construct_object(self, node, deep) 88 data = constructor(self, node) 89 else: ---> 90 data = constructor(self, tag_suffix, node) 91 if isinstance(data, types.GeneratorType): 92 generator = data /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in multi_constructor_obj(loader, tag_suffix, node) 356 yaml_src = yaml.serialize(node) 357 construct_mapping(node) --> 358 mapping = loader.construct_mapping(node) 359 360 assert hasattr(mapping, 'keys') /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in construct_mapping(self, node, deep) 206 if isinstance(node, MappingNode): 207 self.flatten_mapping(node) --> 208 return BaseConstructor.construct_mapping(self, node, deep=deep) 209 210 def construct_yaml_null(self, node): /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in construct_mapping(self, node, deep) 131 raise ConstructorError("while constructing a mapping", node.start_mark, 132 "found unacceptable key (%s)" % exc, key_node.start_mark) --> 133 value = self.construct_object(value_node, deep=deep) 134 mapping[key] = value 135 return mapping /usr/lib/python2.7/dist-packages/yaml/constructor.pyc in construct_object(self, node, deep) 88 data = constructor(self, node) 89 else: ---> 90 data = constructor(self, tag_suffix, node) 91 if isinstance(data, types.GeneratorType): 92 generator = data /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in multi_constructor_obj(loader, tag_suffix, node) 370 callable = eval(tag_suffix) 371 else: --> 372 callable = try_to_import(tag_suffix) 373 rval = Proxy(callable=callable, yaml_src=yaml_src, positionals=(), 374 keywords=mapping) /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in try_to_import(tag_suffix) 297 base_msg += ' but could import %s' % modulename 298 reraise_as(ImportError(base_msg + '. Original exception: ' --> 299 + str(e))) 300 j += 1 301 try: /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/utils/exc.pyc in reraise_as(new_exc) 88 new_exc.__cause__ = orig_exc_value 89 new_exc.reraised = True ---> 90 six.reraise(type(new_exc), new_exc, orig_exc_traceback) /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in try_to_import(tag_suffix) 290 modulename = '.'.join(pcomponents[:j]) 291 try: --> 292 exec('import %s' % modulename) 293 except Exception: 294 base_msg = 'Could not import %s' % modulename /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/config/yaml_parse.pyc in () /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/models/softmax_regression.py in () 13 __maintainer__ = "LISA Lab" 14 ---> 15 from pylearn2.models import mlp 16 17 class SoftmaxRegression(mlp.MLP): /home/amila/Documents/Algorithm/git/pylearn2/pylearn2/models/mlp.py in () 20 from theano.gof.op import get_debug_values 21 from theano.sandbox.rng_mrg import MRG_RandomStreams ---> 22 from theano.sandbox.cuda.dnn import dnn_available, dnn_pool 23 from theano.tensor.signal.downsample import max_pool_2d 24 import theano.tensor as T ImportError: Could not import pylearn2.models.softmax_regression but could import pylearn2.models. Original exception: No module named dnn Original exception: ImportError: No module named dnn From steve+comp.lang.python at pearwood.info Fri Feb 27 06:54:39 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 27 Feb 2015 22:54:39 +1100 Subject: Newbie question about text encoding References: <201502241507.t1OF7aUm018883@fido.openend.se> <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> Dave Angel wrote: > On 02/27/2015 12:58 AM, Steven D'Aprano wrote: >> Dave Angel wrote: >> >>> (Although I believe Seymour Cray was quoted as saying that virtual >>> memory is a crock, because "you can't fake what you ain't got.") >> >> If I recall correctly, disk access is about 10000 times slower than RAM, >> so virtual memory is *at least* that much slower than real memory. >> > > It's so much more complicated than that, that I hardly know where to > start. [snip technical details] As interesting as they were, none of those details will make swap faster, hence my comment that virtual memory is *at least* 10000 times slower than RAM. -- Steven From davea at davea.name Fri Feb 27 09:02:37 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 09:02:37 -0500 Subject: Newbie question about text encoding In-Reply-To: <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> References: <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54F078FD.5030801@davea.name> On 02/27/2015 06:54 AM, Steven D'Aprano wrote: > Dave Angel wrote: > >> On 02/27/2015 12:58 AM, Steven D'Aprano wrote: >>> Dave Angel wrote: >>> >>>> (Although I believe Seymour Cray was quoted as saying that virtual >>>> memory is a crock, because "you can't fake what you ain't got.") >>> >>> If I recall correctly, disk access is about 10000 times slower than RAM, >>> so virtual memory is *at least* that much slower than real memory. >>> >> >> It's so much more complicated than that, that I hardly know where to >> start. > > [snip technical details] > > As interesting as they were, none of those details will make swap faster, > hence my comment that virtual memory is *at least* 10000 times slower than > RAM. > The term "virtual memory" is used for many aspects of the modern memory architecture. But I presume you're using it in the sense of "running in a swapfile" as opposed to running in physical RAM. Yes, a page fault takes on the order of 10,000 times as long as an access to a location in L1 cache. I suspect it's a lot smaller though if the swapfile is on an SSD drive. The first byte is that slow. But once the fault is resolved, the nearby bytes are in physical memory, and some of them are in L3, L2, and L1. So you're not running in the swapfile any more. And even when you run off the end of the page, fetching the sequentially adjacent page from a hard disk is much faster. And if the disk has well designed buffering, faster yet. The OS tries pretty hard to keep the swapfile unfragmented. The trick is to minimize the number of page faults, especially to random locations. If you're getting lots of them, it's called thrashing. There are tools to help with that. To minimize page faults on code, linking with a good working-set-tuner can help, though I don't hear of people bothering these days. To minimize page faults on data, choosing one's algorithm carefully can help. For example, in scanning through a typical matrix, row order might be adjacent locations, while column order might be scattered. Not really much different than reading a text file. If you can arrange to process it a line at a time, rather than reading the whole file into memory, you generally minimize your round-trips to disk. And if you need to randomly access it, it's quite likely more efficient to memory map it, in which case it temporarily becomes part of the swapfile system. -- DaveA From rosuav at gmail.com Fri Feb 27 09:22:15 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 01:22:15 +1100 Subject: Newbie question about text encoding In-Reply-To: <54F078FD.5030801@davea.name> References: <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, Feb 28, 2015 at 1:02 AM, Dave Angel wrote: > The term "virtual memory" is used for many aspects of the modern memory > architecture. But I presume you're using it in the sense of "running in a > swapfile" as opposed to running in physical RAM. Given that this started with a quote about "you can't fake what you ain't got", I would say that, yes, this refers to using hard disk to provide more RAM. If you're trying to use the pagefile/swapfile as if it's more memory ("I have 256MB of memory, but 10GB of swap space, so that's 10GB of memory!"), then yes, these performance considerations are huge. But suppose you need to run a program that's larger than your available RAM. On MS-DOS, sometimes you'd need to work with program overlays (a concept borrowed from older systems, but ones that I never worked on, so I'm going back no further than DOS here). You get a *massive* complexity hit the instant you start using them, whether your program would have been able to fit into memory on some systems or not. Just making it possible to have only part of your code in memory places demands on your code that you, the programmer, have to think about. With virtual memory, though, you just write your code as if it's all in memory, and some of it may, at some times, be on disk. Less code to debug = less time spent debugging. The performance question is largely immaterial (you'll be using the disk either way), but the savings on complexity are tremendous. And then when you do find yourself running on a system with enough RAM? No code changes needed, and full performance. That's where virtual memory shines. It's funny how the world changes, though. Back in the 90s, virtual memory was the key. No home computer ever had enough RAM. Today? A home-grade PC could easily have 16GB... and chances are you don't need all of that. So we go for the opposite optimization: disk caching. Apart from when I rebuild my "Audio-Only Frozen" project [1] and the caches get completely blasted through, heaps and heaps of my work can be done inside the disk cache. Hey, Sikorsky, got any files anywhere on the hard disk matching *Pastel*.iso case insensitively? *chug chug chug* Nope. Okay. Sikorsky, got any files matching *Pas5*.iso case insensitively? *zip* Yeah, here it is. I didn't tell the first search to hold all that file system data in memory; the hard drive controller managed it all for me, and I got the performance benefit. Same as the above: the main benefit is that this sort of thing requires zero application code complexity. It's all done in a perfectly generic way at a lower level. ChrisA From davea at davea.name Fri Feb 27 10:24:55 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 10:24:55 -0500 Subject: Newbie question about text encoding In-Reply-To: References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: <54F08C47.1080007@davea.name> On 02/27/2015 09:22 AM, Chris Angelico wrote: > On Sat, Feb 28, 2015 at 1:02 AM, Dave Angel wrote: >> The term "virtual memory" is used for many aspects of the modern memory >> architecture. But I presume you're using it in the sense of "running in a >> swapfile" as opposed to running in physical RAM. > > Given that this started with a quote about "you can't fake what you > ain't got", I would say that, yes, this refers to using hard disk to > provide more RAM. > > If you're trying to use the pagefile/swapfile as if it's more memory > ("I have 256MB of memory, but 10GB of swap space, so that's 10GB of > memory!"), then yes, these performance considerations are huge. But > suppose you need to run a program that's larger than your available > RAM. On MS-DOS, sometimes you'd need to work with program overlays (a > concept borrowed from older systems, but ones that I never worked on, > so I'm going back no further than DOS here). You get a *massive* > complexity hit the instant you start using them, whether your program > would have been able to fit into memory on some systems or not. Just > making it possible to have only part of your code in memory places > demands on your code that you, the programmer, have to think about. > With virtual memory, though, you just write your code as if it's all > in memory, and some of it may, at some times, be on disk. Less code to > debug = less time spent debugging. The performance question is largely > immaterial (you'll be using the disk either way), but the savings on > complexity are tremendous. And then when you do find yourself running > on a system with enough RAM? No code changes needed, and full > performance. That's where virtual memory shines. > > It's funny how the world changes, though. Back in the 90s, virtual > memory was the key. No home computer ever had enough RAM. Today? A > home-grade PC could easily have 16GB... and chances are you don't need > all of that. So we go for the opposite optimization: disk caching. > Apart from when I rebuild my "Audio-Only Frozen" project [1] and the > caches get completely blasted through, heaps and heaps of my work can > be done inside the disk cache. Hey, Sikorsky, got any files anywhere > on the hard disk matching *Pastel*.iso case insensitively? *chug chug > chug* Nope. Okay. Sikorsky, got any files matching *Pas5*.iso case > insensitively? *zip* Yeah, here it is. I didn't tell the first search > to hold all that file system data in memory; the hard drive controller > managed it all for me, and I got the performance benefit. Same as the > above: the main benefit is that this sort of thing requires zero > application code complexity. It's all done in a perfectly generic way > at a lower level. In 1973, I did manual swapping to an external 8k ramdisk. It was a box that sat on the floor and contained 8k of core memory (not semiconductor). The memory was non-volatile, so it contained the working copy of my code. Then I built a small swapper that would bring in the set of routines currently needed. My onboard RAM (semiconductor) was 1.5k, which had to hold the swapper, the code, and the data. I was writing a GPS system for shipboard use, and the final version of the code had to fit entirely in EPROM, 2k of it. But debugging EPROM code is a pain, since every small change took half an hour to make new chips. Later, I built my first PC with 512k of RAM, and usually used much of it as a ramdisk, since programs didn't use nearly that amount. -- DaveA From alister.nospam.ware at ntlworld.com Fri Feb 27 11:00:09 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Fri, 27 Feb 2015 16:00:09 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, 28 Feb 2015 01:22:15 +1100, Chris Angelico wrote: > > If you're trying to use the pagefile/swapfile as if it's more memory ("I > have 256MB of memory, but 10GB of swap space, so that's 10GB of > memory!"), then yes, these performance considerations are huge. But > suppose you need to run a program that's larger than your available RAM. > On MS-DOS, sometimes you'd need to work with program overlays (a concept > borrowed from older systems, but ones that I never worked on, so I'm > going back no further than DOS here). You get a *massive* complexity hit > the instant you start using them, whether your program would have been > able to fit into memory on some systems or not. Just making it possible > to have only part of your code in memory places demands on your code > that you, the programmer, have to think about. With virtual memory, > though, you just write your code as if it's all in memory, and some of > it may, at some times, be on disk. Less code to debug = less time spent > debugging. The performance question is largely immaterial (you'll be > using the disk either way), but the savings on complexity are > tremendous. And then when you do find yourself running on a system with > enough RAM? No code changes needed, and full performance. That's where > virtual memory shines. > ChrisA I think there is a case for bringing back the overlay file, or at least loading larger programs in sections only loading the routines as they are required could speed up the start time of many large applications. examples libre office, I rarely need the mail merge function, the word count and may other features that could be added into the running application on demand rather than all at once. obviously with large memory & virtual mem there is no need to un-install them once loaded. -- Ralph's Observation: It is a mistake to let any mechanical object realise that you are in a hurry. From rosuav at gmail.com Fri Feb 27 11:12:16 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 03:12:16 +1100 Subject: Newbie question about text encoding In-Reply-To: References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, Feb 28, 2015 at 3:00 AM, alister wrote: > I think there is a case for bringing back the overlay file, or at least > loading larger programs in sections > only loading the routines as they are required could speed up the start > time of many large applications. > examples libre office, I rarely need the mail merge function, the word > count and may other features that could be added into the running > application on demand rather than all at once. Downside of that is twofold: firstly the complexity that I already mentioned, and secondly you pay the startup cost on first usage. So you might get into the program a bit faster, but as soon as you go to any feature you didn't already hit this session, the program pauses for a bit and loads it. Sometimes startup cost is the best time to do this sort of thing. Of course, there is an easy way to implement exactly what you're asking for: use separate programs for everything, instead of expecting a megantic office suite[1] to do everything for you. Just get yourself a nice simple text editor, then invoke other programs - maybe from a terminal, or maybe from within the editor - to do the rest of the work. A simple disk cache will mean that previously-used programs start up quickly. ChrisA [1] It's slightly less bloated than the gigantic office suite sold by a top-end software company. From alister.nospam.ware at ntlworld.com Fri Feb 27 11:45:34 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Fri, 27 Feb 2015 16:45:34 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, 28 Feb 2015 03:12:16 +1100, Chris Angelico wrote: > On Sat, Feb 28, 2015 at 3:00 AM, alister > wrote: >> I think there is a case for bringing back the overlay file, or at least >> loading larger programs in sections only loading the routines as they >> are required could speed up the start time of many large applications. >> examples libre office, I rarely need the mail merge function, the word >> count and may other features that could be added into the running >> application on demand rather than all at once. > > Downside of that is twofold: firstly the complexity that I already > mentioned, and secondly you pay the startup cost on first usage. So you > might get into the program a bit faster, but as soon as you go to any > feature you didn't already hit this session, the program pauses for a > bit and loads it. Sometimes startup cost is the best time to do this > sort of thing. > If the modules are small enough this may not be noticeable but yes I do accept there may be delays on first usage. As to the complexity it has been my observation that as the memory footprint available to programmers has increase they have become less & less skilled at writing code. of course my time as a professional programmer was over 20 years ago on 8 bit micro controllers with 8k of ROM (eventually, original I only had 2k to play with) & 128 Bytes (yes bytes!) of RAM so I am very out of date. I now play with python because it is so much less demanding of me which probably makes me just a guilty :-) > Of course, there is an easy way to implement exactly what you're asking > for: use separate programs for everything, instead of expecting a > megantic office suite[1] to do everything for you. Just get yourself a > nice simple text editor, then invoke other programs - maybe from a > terminal, or maybe from within the editor - to do the rest of the work. > A simple disk cache will mean that previously-used programs start up > quickly. Libre office was sighted as just one example Video editing suites are another that could be used as an example (perhaps more so, does the rendering engine need to be loaded until you start generating the output? a small delay here would be insignificant) > > ChrisA > > [1] It's slightly less bloated than the gigantic office suite sold by a > top-end software company. -- You don't sew with a fork, so I see no reason to eat with knitting needles. -- Miss Piggy, on eating Chinese Food From rosuav at gmail.com Fri Feb 27 12:45:04 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 04:45:04 +1100 Subject: Newbie question about text encoding In-Reply-To: References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, Feb 28, 2015 at 3:45 AM, alister wrote: > On Sat, 28 Feb 2015 03:12:16 +1100, Chris Angelico wrote: > >> On Sat, Feb 28, 2015 at 3:00 AM, alister >> wrote: >>> I think there is a case for bringing back the overlay file, or at least >>> loading larger programs in sections only loading the routines as they >>> are required could speed up the start time of many large applications. >>> examples libre office, I rarely need the mail merge function, the word >>> count and may other features that could be added into the running >>> application on demand rather than all at once. >> >> Downside of that is twofold: firstly the complexity that I already >> mentioned, and secondly you pay the startup cost on first usage. So you >> might get into the program a bit faster, but as soon as you go to any >> feature you didn't already hit this session, the program pauses for a >> bit and loads it. Sometimes startup cost is the best time to do this >> sort of thing. >> > If the modules are small enough this may not be noticeable but yes I do > accept there may be delays on first usage. > > As to the complexity it has been my observation that as the memory > footprint available to programmers has increase they have become less & > less skilled at writing code. Perhaps, but on the other hand, the skill of squeezing code into less memory is being replaced by other skills. We can write code that takes the simple/dumb approach, let it use an entire megabyte of memory, and not care about the cost... and we can write that in an hour, instead of spending a week fiddling with it. Reducing the development cycle time means we can add all sorts of cool features to a program, all while the original end user is still excited about it. (Of course, a comparison between today's World Wide Web and that of the 1990s suggests that these cool features aren't necessarily beneficial, but still, we have the option of foregoing austerity.) > Video editing suites are another that could be used as an example > (perhaps more so, does the rendering engine need to be loaded until you > start generating the output? a small delay here would be insignificant) Hmm, I'm not sure that's actually a big deal, because your *data* will dwarf the code. I can fire up sox and avconv, both fairly large programs, and their code will all sit comfortably in memory; but then they get to work on my data, and suddenly my hard disk is chewing through 91GB of content. Breaking up avconv into a dozen pieces wouldn't make a dent in 91GB. ChrisA From invalid at invalid.invalid Fri Feb 27 12:46:01 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 27 Feb 2015 17:46:01 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2015-02-27, Steven D'Aprano wrote: > Dave Angel wrote: > >> On 02/27/2015 12:58 AM, Steven D'Aprano wrote: >>> Dave Angel wrote: >>> >>>> (Although I believe Seymour Cray was quoted as saying that virtual >>>> memory is a crock, because "you can't fake what you ain't got.") >>> >>> If I recall correctly, disk access is about 10000 times slower than RAM, >>> so virtual memory is *at least* that much slower than real memory. >>> >> >> It's so much more complicated than that, that I hardly know where to >> start. > > [snip technical details] > > As interesting as they were, none of those details will make swap faster, > hence my comment that virtual memory is *at least* 10000 times slower than > RAM. Nonsense. On all of my machines, virtual memory _is_ RAM almost all of the time. I don't do the type of things that force the usage of swap. -- Grant Edwards grant.b.edwards Yow! ... I want FORTY-TWO at TRYNEL FLOATATION SYSTEMS gmail.com installed within SIX AND A HALF HOURS!!! From invalid at invalid.invalid Fri Feb 27 12:47:10 2015 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 27 Feb 2015 17:47:10 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241524.t1OFO09k022270@fido.openend.se> <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2015-02-27, Grant Edwards wrote: > On 2015-02-27, Steven D'Aprano wrote: Dave Angel wrote: >>> On 02/27/2015 12:58 AM, Steven D'Aprano wrote: Dave Angel wrote: >>>> >>>>> (Although I believe Seymour Cray was quoted as saying that virtual >>>>> memory is a crock, because "you can't fake what you ain't got.") >>>> >>>> If I recall correctly, disk access is about 10000 times slower than RAM, >>>> so virtual memory is *at least* that much slower than real memory. >>>> >>> It's so much more complicated than that, that I hardly know where to >>> start. >> >> [snip technical details] >> >> As interesting as they were, none of those details will make swap faster, >> hence my comment that virtual memory is *at least* 10000 times slower than >> RAM. > > Nonsense. On all of my machines, virtual memory _is_ RAM almost all > of the time. I don't do the type of things that force the usage of > swap. And on some of the embedded systems I work on, _all_ virtual memory is RAM 100.000% of the time. -- Grant Edwards grant.b.edwards Yow! Don't SANFORIZE me!! at gmail.com From pkpearson at nowhere.invalid Fri Feb 27 12:55:19 2015 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 27 Feb 2015 17:55:19 GMT Subject: Gaussian process regression References: <5d2cd3fd-9d63-4e79-af1c-965d50145801@googlegroups.com> Message-ID: On Thu, 26 Feb 2015 09:59:45 -0800 (PST), jaykim.huijae at gmail.com wrote: > > I am trying to use Gaussian process regression for Near Infrared > spectra. I have reference data(spectra), concentrations of reference > data and sample data, and I am trying to predict concentrations of > sample data. Here is my code. > > from sklearn.gaussian_process import GaussianProcess > gp = GaussianProcess() > gp.fit(reference, concentration) > concentration_pred = gp.predict(sample) [snip] I'm sorry you're not getting help from this normally very helpful group. I'd guess that's because nobody here uses sklearn. Where did you get sklearn? Is it possible that there's an sklearn forum somewhere? I've seen many of this group's regular participants go to great lengths to help people with specialized problems, but for one of those people to help with your problem, he or she would have to find and install sklearn and learn enough about it to generate data sets on which to exercise the code you've provided. That's a lot to ask. Can you lower the activation barrier? -- To email me, substitute nowhere->runbox, invalid->com. From fabien.maussion at gmail.com Fri Feb 27 13:33:32 2015 From: fabien.maussion at gmail.com (Fabien) Date: Fri, 27 Feb 2015 19:33:32 +0100 Subject: Gaussian process regression References: <5d2cd3fd-9d63-4e79-af1c-965d50145801@googlegroups.com> Message-ID: On 27.02.2015 18:55, Peter Pearson wrote: > On Thu, 26 Feb 2015 09:59:45 -0800 (PST),jaykim.huijae at gmail.com wrote: >> > >> >I am trying to use Gaussian process regression for Near Infrared >> >spectra. I have reference data(spectra), concentrations of reference >> >data and sample data, and I am trying to predict concentrations of >> >sample data. Here is my code. >> > >> >from sklearn.gaussian_process import GaussianProcess >> >gp = GaussianProcess() >> >gp.fit(reference, concentration) >> >concentration_pred = gp.predict(sample) > [snip] > > I'm sorry you're not getting help from this normally very helpful group. > I'd guess that's because nobody here uses sklearn. Where did you get > sklearn? Is it possible that there's an sklearn forum somewhere? http://blog.gmane.org/gmane.comp.python.scikit-learn Cheers, Fabien From auriocus at gmx.de Fri Feb 27 13:55:44 2015 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 27 Feb 2015 19:55:44 +0100 Subject: Parallelization of Python on GPU? In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> Message-ID: Am 26.02.15 um 06:53 schrieb John Ladasky: > Thanks for the various links, Ethan. I have encountered PyCUDA before, but not the other options. > > So far, I'm not seeing code examples which appear to do what I would like, which is simply to farm out one Python process to one GPU core. The examples all appear to parallelize array operations. I know, that's the easier way to break up a task. > > I may have to bite the bullet and learn how to use this: > > http://mklab.iti.gr/project/GPU-LIBSVM > If you can get this to run on your machine, it will surely outperform any efforts what you can do with a python-CUDA bridge on your own. GPU programming is hard, and efficient GPU programming is really hard. To get an impression, this talk shows how some changes to an OpenCL program can improve the speed by 60x compared to a naive implementation: http://web.archive.org/web/20101217181349/http://developer.amd.com/zones/OpenCLZone/Events/assets/Optimizations-ImageConvolution1.pdf Christian From python at mrabarnett.plus.com Fri Feb 27 14:14:00 2015 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 27 Feb 2015 19:14:00 +0000 Subject: Newbie question about text encoding In-Reply-To: References: <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: <54F0C1F8.50209@mrabarnett.plus.com> On 2015-02-27 16:45, alister wrote: > On Sat, 28 Feb 2015 03:12:16 +1100, Chris Angelico wrote: > >> On Sat, Feb 28, 2015 at 3:00 AM, alister >> wrote: >>> I think there is a case for bringing back the overlay file, or at least >>> loading larger programs in sections only loading the routines as they >>> are required could speed up the start time of many large applications. >>> examples libre office, I rarely need the mail merge function, the word >>> count and may other features that could be added into the running >>> application on demand rather than all at once. >> >> Downside of that is twofold: firstly the complexity that I already >> mentioned, and secondly you pay the startup cost on first usage. So you >> might get into the program a bit faster, but as soon as you go to any >> feature you didn't already hit this session, the program pauses for a >> bit and loads it. Sometimes startup cost is the best time to do this >> sort of thing. >> > If the modules are small enough this may not be noticeable but yes I do > accept there may be delays on first usage. > I suppose you could load the basic parts first so that the user can start working, and then load the additional features in the background. > As to the complexity it has been my observation that as the memory > footprint available to programmers has increase they have become less & > less skilled at writing code. > > of course my time as a professional programmer was over 20 years ago on 8 > bit micro controllers with 8k of ROM (eventually, original I only had 2k > to play with) & 128 Bytes (yes bytes!) of RAM so I am very out of date. > > I now play with python because it is so much less demanding of me which > probably makes me just a guilty :-) > >> Of course, there is an easy way to implement exactly what you're asking >> for: use separate programs for everything, instead of expecting a >> megantic office suite[1] to do everything for you. Just get yourself a >> nice simple text editor, then invoke other programs - maybe from a >> terminal, or maybe from within the editor - to do the rest of the work. >> A simple disk cache will mean that previously-used programs start up >> quickly. > Libre office was sighted as just one example > Video editing suites are another that could be used as an example > (perhaps more so, does the rendering engine need to be loaded until you > start generating the output? a small delay here would be insignificant) >> >> ChrisA >> >> [1] It's slightly less bloated than the gigantic office suite sold by a >> top-end software company. > From simon+python at bleah.co.uk Fri Feb 27 15:06:25 2015 From: simon+python at bleah.co.uk (Simon Ward) Date: Fri, 27 Feb 2015 20:06:25 +0000 Subject: Python Worst Practices In-Reply-To: <85h9u82wq1.fsf@benfinney.id.au> References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> <85h9u82wq1.fsf@benfinney.id.au> Message-ID: <757f69ee-b66b-426c-98e2-3b76c11f6075@email.android.com> On 26 February 2015 21:23:34 GMT+00:00, Ben Finney wrote: >Simon Ward writes: >> 0 = success and non-zero = failure is the meme established, rather >> than 0 = true, non-zero = false. > >That is not the case: the commands ?true? (returns value 0) and ?false? >(returns value 1) are long established in Unix. So that *is* part of >the >meme I'm describing. I mentioned the true and false. OK, so it's a meme, but it's based on a false (pun intended) understanding of exit status codes. That success evaluates to true and failure evaluates to false does not mean the values of truth and falseness are inverted. No programming language other than that provided by system shells I have used evaluates 0 to true. The shell case is unique because it typically runs processes and has to deal with exit status codes. It feels quite natural to me that, if COMMAND completes successfully (exit code 0) then writing "if COMMAND ; then : true bit ; else : false bit ; fi" should execute the true bit, but that doesn't mean all boolean logic is inverted. true is define to return 0, or no error. false is defined to return 1, or error. They are just commands though, and the exit codes are still exit codes, not boolean values. >> None of the above is a good reason to use error *or* success return >> values in Python--use exceptions!--but may be encountered when >running >> other processes. > >Right. But likewise, don't deny that ?true == 0? and ?false == >non-zero? >has a wide acceptance in the programming community too. I can't deny that, but I can state what things really mean and point out why everybody is doing it wrong. Simon From catalinfest at gmail.com Fri Feb 27 15:11:58 2015 From: catalinfest at gmail.com (blue) Date: Fri, 27 Feb 2015 12:11:58 -0800 (PST) Subject: function inclusion problem In-Reply-To: References: Message-ID: <01e7effc-7cf2-4e27-a8df-d1fc86613f01@googlegroups.com> On Wednesday, February 11, 2015 at 1:38:12 AM UTC+2, vlya... at gmail.com wrote: > I defined function Fatalln in "mydef.py" and it works fine if i call it from "mydef.py", but when i try to call it from "test.py" in the same folder: > import mydef > ... > Fatalln "my test" > i have NameError: name 'Fatalln' is not defined > I also tried include('mydef.py') with the same result... > What is the right syntax? > Thanks ...try to set your python utf-8 encode . and read the FAQ or python manual From simon+python at bleah.co.uk Fri Feb 27 15:23:26 2015 From: simon+python at bleah.co.uk (Simon Ward) Date: Fri, 27 Feb 2015 20:23:26 +0000 Subject: Python Worst Practices In-Reply-To: <757f69ee-b66b-426c-98e2-3b76c11f6075@email.android.com> References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> <85h9u82wq1.fsf@benfinney.id.au> <757f69ee-b66b-426c-98e2-3b76c11f6075@email.android.com> Message-ID: <6e99403d-1dd5-4abb-9441-a648af10f1bc@email.android.com> On 27 February 2015 20:06:25 GMT+00:00, I wrote: >I mentioned the true and false. OK, so it's a meme, but it's based on a >false (pun intended) understanding of exit status codes. That success >evaluates to true and failure evaluates to false does not mean the >values of truth and falseness are inverted. No programming language >other than that provided by system shells I have used evaluates 0 to >true. I hope the following examples from bash illustrate this: $ (( 0 )) && echo success $ (( 1 )) && echo success success $ (( 0 )) ; echo $? 1 $ (( 1 )) ; echo $? 0 Simon From travisgriggs at gmail.com Fri Feb 27 15:41:46 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Fri, 27 Feb 2015 12:41:46 -0800 Subject: Pyston 0.3 self-hosting In-Reply-To: <54ed620d$0$11100$c3e8da3@news.astraweb.com> References: <54ed620d$0$11100$c3e8da3@news.astraweb.com> Message-ID: > On Feb 24, 2015, at 9:47 PM, Steven D'Aprano wrote: > > Pyston 0.3, the latest version of a new high-performance Python > implementation, has reached self-hosting sufficiency: > > > http://blog.pyston.org/2015/02/24/pyston-0-3-self-hosting-sufficiency/ > Does it do python3.4 yet? From ethan at stoneleaf.us Fri Feb 27 15:51:26 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 27 Feb 2015 12:51:26 -0800 Subject: Pyston 0.3 self-hosting In-Reply-To: References: <54ed620d$0$11100$c3e8da3@news.astraweb.com> Message-ID: <54F0D8CE.2080605@stoneleaf.us> On 02/27/2015 12:41 PM, Travis Griggs wrote: > >> On Feb 24, 2015, at 9:47 PM, Steven D'Aprano wrote: >> >> Pyston 0.3, the latest version of a new high-performance Python >> implementation, has reached self-hosting sufficiency: >> >> >> http://blog.pyston.org/2015/02/24/pyston-0-3-self-hosting-sufficiency/ From their github site [1]: Currently, Pyston targets Python 2.7, only runs on x86_64 platforms, and only has been tested on Ubuntu. Support for more platforms -- along with Python 3 compatibility -- is desired but currently not on the roadmap. -- ~Ethan~ [1] https://github.com/dropbox/pyston -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From davea at davea.name Fri Feb 27 15:52:24 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 15:52:24 -0500 Subject: Newbie question about text encoding In-Reply-To: References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: <54F0D908.3060500@davea.name> On 02/27/2015 11:00 AM, alister wrote: > On Sat, 28 Feb 2015 01:22:15 +1100, Chris Angelico wrote: > >> >> If you're trying to use the pagefile/swapfile as if it's more memory ("I >> have 256MB of memory, but 10GB of swap space, so that's 10GB of >> memory!"), then yes, these performance considerations are huge. But >> suppose you need to run a program that's larger than your available RAM. >> On MS-DOS, sometimes you'd need to work with program overlays (a concept >> borrowed from older systems, but ones that I never worked on, so I'm >> going back no further than DOS here). You get a *massive* complexity hit >> the instant you start using them, whether your program would have been >> able to fit into memory on some systems or not. Just making it possible >> to have only part of your code in memory places demands on your code >> that you, the programmer, have to think about. With virtual memory, >> though, you just write your code as if it's all in memory, and some of >> it may, at some times, be on disk. Less code to debug = less time spent >> debugging. The performance question is largely immaterial (you'll be >> using the disk either way), but the savings on complexity are >> tremendous. And then when you do find yourself running on a system with >> enough RAM? No code changes needed, and full performance. That's where >> virtual memory shines. >> ChrisA > > I think there is a case for bringing back the overlay file, or at least > loading larger programs in sections > only loading the routines as they are required could speed up the start > time of many large applications. > examples libre office, I rarely need the mail merge function, the word > count and may other features that could be added into the running > application on demand rather than all at once. > > obviously with large memory & virtual mem there is no need to un-install > them once loaded. > I can't say how Linux handles it (I'd like to know, but haven't needed to yet), but in Windows (NT, XP, etc), a DLL is not "loaded", but rather mapped. And it's not copied into the swapfile, it's mapped directly from the DLL. The mapping mode is "copy-on-write" which means that read=only portions are swapped directly from the DLL, on first usage, while read-write portions (eg. static/global variables, relocation modifications) are copied on first use to the swap file. I presume EXE's are done the same way, but never had a need to know. If that's the case on the architectures you're talking about, then the problem of slow loading is not triggered by the memory usage, but by lots of initialization code. THAT's what should be deferred for seldom-used portions of code. The main point of a working-set-tuner is to group sections of code together that are likely to be used together. To take an extreme case, all the fatal exception handlers should be positioned adjacent to each other in linear memory, as it's unlikely that any of them will be needed, and the code takes up no time or space in physical memory. Also (in Windows), a DLL can be pre-relocated, so that it has a preferred address to be loaded into memory. If that memory is available when it gets loaded (actually mapped), then no relocation needs to happen, which saves time and swap space. In the X86 architecture, most code is self-relocating, everything is relative. But references to other DLL's and jump tables were absolute, so they needed to be relocated at load time, when final locations were nailed down. Perhaps the authors of bloated applications have forgotten how to do these, as the defaults in the linker puts all DLL's in the same location, meaning all but the first will need relocating. But system DLL's are (were) each given unique addresses. On one large project, I added the build step of assigning these base addresses. Each DLL had to start on a 64k boundary, and I reserved some fractional extra space between them in case one would grow. Then every few months, we double-checked that they didn't overlap, and if necessary adjusted the start addresses. We didn't just automatically assign closest addresses, because frequently some of the DLL's would be updated independently of the others. -- DaveA From rosuav at gmail.com Fri Feb 27 16:04:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 08:04:44 +1100 Subject: Newbie question about text encoding In-Reply-To: <54F0D908.3060500@davea.name> References: <201502241620.t1OGKf4n002146@fido.openend.se> <54ECB134.5090304@davea.name> <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> <54F0D908.3060500@davea.name> Message-ID: On Sat, Feb 28, 2015 at 7:52 AM, Dave Angel wrote: > If that's the case on the architectures you're talking about, then the > problem of slow loading is not triggered by the memory usage, but by lots of > initialization code. THAT's what should be deferred for seldom-used > portions of code. s/should/can/ It's still not a clear case of "should", as it's all a big pile of trade-offs. A few weeks ago I made a very deliberate change to a process to force some code to get loaded and initialized earlier, to prevent an unexpected (and thus surprising) slowdown on first use. (It was, in fact, a Python 'import' statement, so all I had to do was add a dummy import in the main module - with, of course, a comment making it clear that this was necessary, even though the name wasn't used.) But yes, seldom-used code can definitely have its initialization deferred if you need to speed up startup. ChrisA From travisgriggs at gmail.com Fri Feb 27 16:21:27 2015 From: travisgriggs at gmail.com (Travis Griggs) Date: Fri, 27 Feb 2015 13:21:27 -0800 Subject: Python Worst Practices In-Reply-To: References: Message-ID: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> > On Feb 25, 2015, at 12:45 PM, Mark Lawrence wrote: > > http://www.slideshare.net/pydanny/python-worst-practices > > Any that should be added to this list? Any that be removed as not that bad? I read ?em. I thought they were pretty good, some more than others. And I learned some things. I especially liked the first one, since I?ve struggled with that one a bunch. In the context of ?hey, accept Python for what it is?, I agree greatly with it. Memorize the builtins, and stay the heck away from them. I?ve been burned many times because I stored some bytes in a binding call, er, uh, ?bytes?. And having mentored some people learning Python in the early stages, any explanation other than ?Thou Shalt Never Use These Holy Words? just confuses people. That said? If I were giving a talk at SPLASH (or some other suitable polyglot conference), I might do one called ?Language Design Worst Practices?. One of my first slides might be titled: Abuse Common Tokens in Confusing Ways * Make your language have a lot of keywords. Enough to make memorizing them ALL unlikely, requiring constant visits to your documentation * Make sure said keywords are many of the obvious words programmers would use in their applications (map, object, bytes, dir, etc) * Design your syntax so that you can?t disambiguate them contextually between bind and reference * Be sure to use it in a late bound language where no warnings will be provided about the mistake you?re making at authorship time, deferring the educational experience to sundry run times In my examples column of this bad practice, I?d put Python of course. :) I do like Python, and I accept it for what it is, so no one needs to jump forward as a Holy Python See to convert me to the truth. I also know that with most other languages, that first slide wouldn?t need to be one of the prominent ?worst practices? slide. From rosuav at gmail.com Fri Feb 27 16:35:20 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 08:35:20 +1100 Subject: Python Worst Practices In-Reply-To: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> Message-ID: On Sat, Feb 28, 2015 at 8:21 AM, Travis Griggs wrote: > > I do like Python, and I accept it for what it is, so no one needs to jump forward as a Holy Python See to convert me to the truth. I also know that with most other languages, that first slide wouldn?t need to be one of the prominent ?worst practices? slide. > This is the thing about design. We are happy to rant about the problems in the things we love most... and it doesn't mean we don't love them. A passing comment in the famous "PHP: a fractal of bad design" article says: "Side observation: I loooove Python. I will also happily talk your ear off complaining about it, if you really want me to."; for myself, I often say that Python is the second-best programming language in the world, but will still rant about its problems. And if I ever start building my own programming language, I would be looking at all the ones I know, and making objective evaluations of what's good and bad about them. (And then probably abandoning the project early on, because there are established languages that are sufficiently close to what I want that it's just not worth making a new one.) Know what you hate about the things you love, and know what you love about the things you hate. ChrisA From davea at davea.name Fri Feb 27 16:37:17 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 16:37:17 -0500 Subject: Python Worst Practices In-Reply-To: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> Message-ID: <54F0E38D.40006@davea.name> On 02/27/2015 04:21 PM, Travis Griggs wrote: > >> On Feb 25, 2015, at 12:45 PM, Mark Lawrence wrote: >> >> http://www.slideshare.net/pydanny/python-worst-practices >> >> Any that should be added to this list? Any that be removed as not that bad? > > I read ?em. I thought they were pretty good, some more than others. And I learned some things. I especially liked the first one, since I?ve struggled with that one a bunch. In the context of ?hey, accept Python for what it is?, I agree greatly with it. Memorize the builtins, and stay the heck away from them. I?ve been burned many times because I stored some bytes in a binding call, er, uh, ?bytes?. And having mentored some people learning Python in the early stages, any explanation other than ?Thou Shalt Never Use These Holy Words? just confuses people. > > That said? > > If I were giving a talk at SPLASH (or some other suitable polyglot conference), I might do one called ?Language Design Worst Practices?. > > One of my first slides might be titled: > > Abuse Common Tokens in Confusing Ways > > * Make your language have a lot of keywords. Enough to make memorizing them ALL unlikely, requiring constant visits to your documentation > * Make sure said keywords are many of the obvious words programmers would use in their applications (map, object, bytes, dir, etc) > * Design your syntax so that you can?t disambiguate them contextually between bind and reference > * Be sure to use it in a late bound language where no warnings will be provided about the mistake you?re making at authorship time, deferring the educational experience to sundry run times > > In my examples column of this bad practice, I?d put Python of course. :) > > I do like Python, and I accept it for what it is, so no one needs to jump forward as a Holy Python See to convert me to the truth. I also know that with most other languages, that first slide wouldn?t need to be one of the prominent ?worst practices? slide. > Right. In C and C++, instead of being the first slide, it'd be the first 3 or 4. Between header file conflicts (especially good because the stdlib itself has many multiply-defined symbols, duplicate header files, and contradictory include path patterns), implib conflicts, and DLL load conflicts, not to mention the notorious win32 directory, which is magically renamed out from under you in 64bit code. (That one is a Windows thing; I wouldn't expect that Linux got caught in that trap) I don't know how much of this has changed in recent years, but I suspect it's gotten much worse, not better. In Python, the error is noticed much later, but it can actually be debugged without tearing out one's hair. Things like print(mymodule.__file__) to figure out where the mymodule is defined. -- DaveA From no.email at nospam.invalid Fri Feb 27 16:40:15 2015 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 27 Feb 2015 13:40:15 -0800 Subject: Future of Pypy? References: <87fv9xdb22.fsf@jester.gateway.pace.com> <54ea7ff4$0$12983$c3e8da3$5496439d@news.astraweb.com> <87zj85bcyu.fsf@jester.gateway.pace.com> <87lhjpb89i.fsf@jester.gateway.pace.com> <87h9udb1eq.fsf@jester.gateway.pace.com> <87bnkkb22u.fsf@jester.gateway.pace.com> <871tlgaxi7.fsf@jester.gateway.pace.com> <87mw43apmf.fsf@jester.gateway.pace.com> <54ec20d0$0$11103$c3e8da3@news.astraweb.com> Message-ID: <878ufj824g.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > An interesting point of view: threading is harmful because it removes > determinism from your program. > http://radar.oreilly.com/2007/01/threads-considered-harmful.html Concurrent programs are inherently nondeterministic because they respond to i/o events that can happen in any order. I looked at the paper cited in that article and it seemed like handwaving. Then it talks about threaded programs being equivalent if they are the same over all interleavings of input, and then goes on about that being horribly difficult to establish. It talked about program inputs as infinite sequences of bits. OK, a standard conceit in mathematical logic is to call an infinite sequence of bits a "real number". So it seems to me that such a proof would just be a theorem about real numbers or sets of real numbers, and freshman calculus classes are already full of proofs like that. The presence of large sets doesn't necessarily make math all that much harder. The test suite for HOL Light actually uses an inaccessible cardinal, if that means anything to you. IOW he says it's difficult and maybe it is, but he doesn't make any attempt to explain why it's difficult, at least once there's some tools (synchronization primitives etc.) to control the concurrency. He seems instead to ignore decades of work going back to Dijkstra and Wirth and those guys. It would be a lot more convincing if he addressed that existing literature and said why it wasn't good enough to help write real programs that work. He then advocates something he calls the "PN model" (processes communicating by message passing) but that seems about the same as what I've heard called communicating sequential processes (CSP), which are the execution model of Erlang and is what I've been using with Python threads and queues. Maybe there's some subtle difference. Anyway there's again plenty of theory about CSP, which are modelled with Pi-calculus (process calculus) which can be interpreted in lambda calculus, so sequential verification techniques are still useful on it. Hmm, I see there's a Wikipedia article "Kahn process networks" about PN networks as mentioned, so I guess I'll look at it. I see it claims a KPN is deterministic on its inputs, while I think CSP's might not be. > Some discussion of the pros and cons of threading: > http://c2.com/cgi/wiki?ThreadsConsideredHarmful This wasn't very informative either. From rosuav at gmail.com Fri Feb 27 16:40:31 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 08:40:31 +1100 Subject: Python Worst Practices In-Reply-To: <54F0E38D.40006@davea.name> References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On Sat, Feb 28, 2015 at 8:37 AM, Dave Angel wrote: > Right. In C and C++, instead of being the first slide, it'd be the first 3 > or 4. Between header file conflicts (especially good because the stdlib > itself has many multiply-defined symbols, duplicate header files, and > contradictory include path patterns) Yeah, Python has some issues with sys.path and how your local module can unexpectedly shadow a stdlib one, but at least the stdlib itself doesn't have any conflicts. I should not ever have to do this dance: #include #undef SOME_SYMBOL #include But sadly, I do. ChrisA From davea at davea.name Fri Feb 27 16:59:04 2015 From: davea at davea.name (Dave Angel) Date: Fri, 27 Feb 2015 16:59:04 -0500 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: <54F0E8A8.4000703@davea.name> On 02/27/2015 04:40 PM, Chris Angelico wrote: > On Sat, Feb 28, 2015 at 8:37 AM, Dave Angel wrote: >> Right. In C and C++, instead of being the first slide, it'd be the first 3 >> or 4. Between header file conflicts (especially good because the stdlib >> itself has many multiply-defined symbols, duplicate header files, and >> contradictory include path patterns) > > Yeah, Python has some issues with sys.path and how your local module > can unexpectedly shadow a stdlib one, but at least the stdlib itself > doesn't have any conflicts. I should not ever have to do this dance: > > #include > #undef SOME_SYMBOL > #include > > But sadly, I do I can remember resorting to #pragmas to print out clues to include order, when recursive includes were masking what on earth was going wrong. Imagine, compile time printing. Fortunately, in Forth and Python, there's much less distinction between compile time and run time. And in Python, lots of metadata and reflection data is left lying around to be inspected in case of confusion. Don't get me wrong, I love C++, even if I don't use it the way the purists would like. But there are many warts that drive me up the wall, and consequently I avoid many features which could improve my C++ life. I haven't found nearly as many in Python. -- DaveA From simon+python at bleah.co.uk Fri Feb 27 17:05:59 2015 From: simon+python at bleah.co.uk (Simon Ward) Date: Fri, 27 Feb 2015 22:05:59 +0000 Subject: Python Worst Practices In-Reply-To: <757f69ee-b66b-426c-98e2-3b76c11f6075@email.android.com> References: <85bnkh5z96.fsf@benfinney.id.au> <85zj814jmb.fsf@benfinney.id.au> <1103e6a0-2183-4641-bf98-2927806c2172@email.android.com> <85h9u82wq1.fsf@benfinney.id.au> <757f69ee-b66b-426c-98e2-3b76c11f6075@email.android.com> Message-ID: On 27 February 2015 20:06:25 GMT+00:00, Simon Ward wrote: > >I mentioned the true and false. OK, so it's a meme, but it's based on a >false (pun intended) understanding of exit status codes. That success >evaluates to true and failure evaluates to false does not mean the >values of truth and falseness are inverted. No programming language >other than that provided by system shells I have used evaluates 0 to >true. > >The shell case is unique because it typically runs processes and has to >deal with exit status codes. It feels quite natural to me that, if >COMMAND completes successfully (exit code 0) then writing "if COMMAND ; >then : true bit ; else : false bit ; fi" should execute the true bit, >but that doesn't mean all boolean logic is inverted. I pointed my partner at this thread, and she said we were using too many of the same words to mean different things. To quote: "you're both using yes or no terms to answer a question that doesn't necessarily need a "yes" or "no" answer. It's still a point of discussion (existentialism next!) but the basic idea seems to be that once the exit code is understood as an error value it is not necessarily true or false. This is what I tried to say originally so I don't disagree. My partner is not a software developer by profession, she deals with food and nutrition and bakes nice cakes. I have encouraged her to learn Python to help with some of her work, which involves calculation of nutritional values and a lot of statistics. Programming idioms should not be lost on her, but she can, and does, point out the bits she is unfamiliar with our has trouble understanding. Simon From alister.nospam.ware at ntlworld.com Fri Feb 27 17:09:39 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Fri, 27 Feb 2015 22:09:39 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Fri, 27 Feb 2015 19:14:00 +0000, MRAB wrote: >> > I suppose you could load the basic parts first so that the user can > start working, and then load the additional features in the background. > quite possible my opinion on this is very fluid it may work for some applications, it probably wouldn't for others. with python it is generally considered good practice to import all modules at the start of a program but there are valid cases fro only importing a module if actually needed. -- Some people have parts that are so private they themselves have no knowledge of them. From alister.nospam.ware at ntlworld.com Fri Feb 27 17:13:08 2015 From: alister.nospam.ware at ntlworld.com (alister) Date: Fri, 27 Feb 2015 22:13:08 +0000 (UTC) Subject: Newbie question about text encoding References: <201502241945.t1OJjshO013092@fido.openend.se> <201502241957.t1OJvrJS015604@fido.openend.se> <00fbd940-52f6-44e2-bf08-b9f35c12e73f@googlegroups.com> <54efc2c8$0$12986$c3e8da3$5496439d@news.astraweb.com> <54f00787$0$12979$c3e8da3$5496439d@news.astraweb.com> <54f05aff$0$12980$c3e8da3$5496439d@news.astraweb.com> <54F078FD.5030801@davea.name> Message-ID: On Sat, 28 Feb 2015 04:45:04 +1100, Chris Angelico wrote: > Perhaps, but on the other hand, the skill of squeezing code into less > memory is being replaced by other skills. We can write code that takes > the simple/dumb approach, let it use an entire megabyte of memory, and > not care about the cost... and we can write that in an hour, instead of > spending a week fiddling with it. Reducing the development cycle time > means we can add all sorts of cool features to a program, all while the > original end user is still excited about it. (Of course, a comparison > between today's World Wide Web and that of the 1990s suggests that these > cool features aren't necessarily beneficial, but still, we have the > option of foregoing austerity.) > > > ChrisA again I am fluid on this 'Clever' programming is often counter productive & unmaintainable, but the again lazy programming can also be just as bad fro this, there is no "one size fits all" solution but the modern environment does make lazy programming very easy. -- After all, all he did was string together a lot of old, well-known quotations. -- H.L. Mencken, on Shakespeare From ian.g.kelly at gmail.com Fri Feb 27 17:37:18 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 27 Feb 2015 15:37:18 -0700 Subject: Python Worst Practices In-Reply-To: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> Message-ID: On Fri, Feb 27, 2015 at 2:21 PM, Travis Griggs wrote: > * Make your language have a lot of keywords. Enough to make memorizing them ALL unlikely, requiring constant visits to your documentation > * Make sure said keywords are many of the obvious words programmers would use in their applications (map, object, bytes, dir, etc) None of those are keywords. Keywords are these: https://docs.python.org/3/reference/lexical_analysis.html#keywords > * Design your syntax so that you can?t disambiguate them contextually between bind and reference Maybe I misunderstand your complaint, but Python draws a sharp syntactic distinction between references and assignment targets: the latter are only ever found to the left of an = in an assignment statement; the former are never found there. There is no reason why an editor should be unable to tell the difference. > * Be sure to use it in a late bound language where no warnings will be provided about the mistake you?re making at authorship time, deferring the educational experience to sundry run times You should lint your code to get warnings about this (and many other things) at authorship time. A good editor should also provide some visual warning when a built-in is shadowed. From jsf80238 at gmail.com Fri Feb 27 19:43:48 2015 From: jsf80238 at gmail.com (Jason Friedman) Date: Fri, 27 Feb 2015 17:43:48 -0700 Subject: requesting you all to please guide me , which tutorials is best to learn redis database In-Reply-To: References: Message-ID: > i want to learn redis database and its use via python , please guide me which tutorials i should be study, so that i can learn it in good way How about https://pypi.python.org/pypi/redis/? From steve+comp.lang.python at pearwood.info Fri Feb 27 20:09:31 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 28 Feb 2015 12:09:31 +1100 Subject: Python Worst Practices References: Message-ID: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Travis Griggs wrote: > If I were giving a talk at SPLASH (or some other suitable polyglot > conference), I might do one called ?Language Design Worst Practices?. > > One of my first slides might be titled: > > Abuse Common Tokens in Confusing Ways > > * Make your language have a lot of keywords. Enough to make memorizing > them ALL unlikely, requiring constant visits to your documentation Is 33 a lot? py> import keyword py> keyword.kwlist ['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield'] > * Make sure said keywords are many of the obvious words programmers would > use in their applications (map, object, bytes, dir, etc) Luckily, Python doesn't make that mistake of making built-ins keywords. That would require making actual changes to the parser each time a new built-in function was added, as well as breaking people's existing code. Fortunately, Python has a much better system: a small set of keywords, very few of which would make useful variable names ("else = 23"), and a much larger set of regular names in a built-in namespace. py> import builtins # use __builtin__ in Python 2 py> sorted(vars(builtins).keys()) ['ArithmeticError', 'AssertionError', ... 'type', 'vars', 'zip'] There's 147 of the built-ins in Python 3.3, although a few of those aren't truly built-in, merely added at interpreter startup. The ability to shadow built-ins is not a bug, it is a feature. It's an amazingly powerful feature, and not particularly newbie-friendly, but *many* things are not easy for newbies to master or avoid abusing. - Code can override, or overload, built-ins, either at the level of an entire module, or inside a particular function. - Modules can offer functions which clash with a built-in name. E.g. reprlib.repr, math.pow. - More importantly, modules can offer stable APIs with no fear that the introduction of a new built-in function will require them to change their function's name. - Which is a special case of a more general benefit, the introduction of a new built-in name does *not* break existing code that already uses that name. Newbies misuse this feature because they still have a wishful-thinking approach to programming. One example of wishful-thinking is the common newbie mistake of wondering why their loop variable never changes: # Toss a coin until you get Tails. x = random.random() while x < 0.5: print "Heads" print "Tails" Isn't it obvious that I want x to get a new random number every time through the loop? I wish the computer understood me so I didn't need to write all the steps out. Likewise: int = 23 n = int("42") Isn't it obvious that the second use of int has to be the built-in function? I wish that the computer would understand from context which one I mean. Other newbie stylistic mistakes which can increase the chance of shadowing errors include: * Too many overly generic variable names like "int" and "str". * Insufficient use of functions and too much top-level code. When they shadow a built-in, they shadow it everywhere. * Excessively large functions that do too much. By the time they reach the end of their 300 line function, they have forgotten that they have already used "list" for a variable name. However, even experienced developers can make this mistake too. Generally speaking, it's trivially easy to recover from. Although if you're doing it *regularly* that might be a hint of deeper problems, e.g. poor variable naming skills, too much top-level code. There's no harm in calling a local variable "id", if you don't use the built-in id() inside that function. That's one of the reasons why functions exist, so that the names you use inside a function are distinct from those outside. > * Design your syntax so that you can?t disambiguate them contextually > between bind and reference Do you have an example of where Python cannot distinguish between a binding operation and a reference? > * Be sure to use it in a late bound language where no warnings will be > provided about the mistake you?re making at authorship time, deferring the > educational experience to sundry run times Python raises a SyntaxError at compile time, not run time, if you try to bind to a keyword: py> global = 23 File "", line 1 global = 23 ^ SyntaxError: invalid syntax -- Steven From sohcahtoa82 at gmail.com Fri Feb 27 20:32:28 2015 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Fri, 27 Feb 2015 17:32:28 -0800 (PST) Subject: Python Worst Practices In-Reply-To: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Friday, February 27, 2015 at 5:09:49 PM UTC-8, Steven D'Aprano wrote: > Travis Griggs wrote: > > > If I were giving a talk at SPLASH (or some other suitable polyglot > > conference), I might do one called "Language Design Worst Practices". > > > > One of my first slides might be titled: > > > > Abuse Common Tokens in Confusing Ways > > > > * Make your language have a lot of keywords. Enough to make memorizing > > them ALL unlikely, requiring constant visits to your documentation > > Is 33 a lot? > > py> import keyword > py> keyword.kwlist > ['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class', > 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', > 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', > 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield'] > > > > * Make sure said keywords are many of the obvious words programmers would > > use in their applications (map, object, bytes, dir, etc) > > Luckily, Python doesn't make that mistake of making built-ins keywords. That > would require making actual changes to the parser each time a new built-in > function was added, as well as breaking people's existing code. > > Fortunately, Python has a much better system: a small set of keywords, very > few of which would make useful variable names ("else = 23"), and a much > larger set of regular names in a built-in namespace. > > > py> import builtins # use __builtin__ in Python 2 > py> sorted(vars(builtins).keys()) > ['ArithmeticError', 'AssertionError', ... 'type', 'vars', 'zip'] > > > There's 147 of the built-ins in Python 3.3, although a few of those aren't > truly built-in, merely added at interpreter startup. > > The ability to shadow built-ins is not a bug, it is a feature. It's an > amazingly powerful feature, and not particularly newbie-friendly, but > *many* things are not easy for newbies to master or avoid abusing. > > - Code can override, or overload, built-ins, either at the level of > an entire module, or inside a particular function. > > - Modules can offer functions which clash with a built-in name. > E.g. reprlib.repr, math.pow. > > - More importantly, modules can offer stable APIs with no fear that > the introduction of a new built-in function will require them to > change their function's name. > > - Which is a special case of a more general benefit, the introduction > of a new built-in name does *not* break existing code that already > uses that name. > > > Newbies misuse this feature because they still have a wishful-thinking > approach to programming. One example of wishful-thinking is the common > newbie mistake of wondering why their loop variable never changes: > > # Toss a coin until you get Tails. > x = random.random() > while x < 0.5: > print "Heads" > print "Tails" > > Isn't it obvious that I want x to get a new random number every time through > the loop? I wish the computer understood me so I didn't need to write all > the steps out. > > > Likewise: > > int = 23 > n = int("42") > > Isn't it obvious that the second use of int has to be the built-in function? > I wish that the computer would understand from context which one I mean. > > Other newbie stylistic mistakes which can increase the chance of shadowing > errors include: > > * Too many overly generic variable names like "int" and "str". > > * Insufficient use of functions and too much top-level code. When they > shadow a built-in, they shadow it everywhere. > > * Excessively large functions that do too much. By the time they reach > the end of their 300 line function, they have forgotten that they > have already used "list" for a variable name. > > > However, even experienced developers can make this mistake too. Generally > speaking, it's trivially easy to recover from. Although if you're doing it > *regularly* that might be a hint of deeper problems, e.g. poor variable > naming skills, too much top-level code. > > There's no harm in calling a local variable "id", if you don't use the > built-in id() inside that function. That's one of the reasons why functions > exist, so that the names you use inside a function are distinct from those > outside. > > > > > * Design your syntax so that you can't disambiguate them contextually > > between bind and reference > > Do you have an example of where Python cannot distinguish between a binding > operation and a reference? > > > > * Be sure to use it in a late bound language where no warnings will be > > provided about the mistake you're making at authorship time, deferring the > > educational experience to sundry run times > > Python raises a SyntaxError at compile time, not run time, if you try to > bind to a keyword: > > py> global = 23 > File "", line 1 > global = 23 > ^ > SyntaxError: invalid syntax > > > > > -- > Steven Very well-said! Just because a feature (In this case, shadowing built-in functions) can be abused or cause problems doesn't mean it's a bad feature. It reminds me of the people that rip on C++'s operator overloading because some people write bad code and implement non-intuitive operators for classes. For example, I've seen someone create a Socket class, then created an operator overload that allowed you to "add" a string to your socket to make the socket send the string, with the result being a status code indicating success or an error. From rosuav at gmail.com Fri Feb 27 20:44:44 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 12:44:44 +1100 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 28, 2015 at 12:32 PM, wrote: > For example, I've seen someone create a Socket class, then created an operator overload that allowed you to "add" a string to your socket to make the socket send the string, with the result being a status code indicating success or an error. > Why not left shift the socket by that string, the result being the original socket? At least that has precedent... ChrisA From python.list at tim.thechases.com Fri Feb 27 22:50:35 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 27 Feb 2015 21:50:35 -0600 Subject: Python Worst Practices In-Reply-To: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20150227215035.0210ea5e@bigbox.christie.dr> On 2015-02-28 12:09, Steven D'Aprano wrote: > > * Make your language have a lot of keywords. Enough to make > > memorizing them ALL unlikely, requiring constant visits to your > > documentation > > Is 33 a lot? > > py> import keyword > py> keyword.kwlist > ['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class', > 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', > 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', > 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', > 'with', 'yield'] A quick google-and-tally for languages and their corresponding number of keywords: C: 33 C#: 77 C++: 86 Java: 50 Lua: 21 PHP: 67 Pascal: 54 Perl: 40 Pike: 37 (Just for you, ChrisA) Python: 31 (2.x) or 33 (3.x) Ruby: 40 So I can't say that Python's all that bad in comparison to most other mainstream languages, with only the austere Lua beating out Python. -tkc From dan at tombstonezero.net Fri Feb 27 23:42:25 2015 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 28 Feb 2015 04:42:25 +0000 (UTC) Subject: Python Worst Practices References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, 28 Feb 2015 12:09:31 +1100, Steven D'Aprano wrote: > There's no harm in calling a local variable "id", if you don't use the > built-in id() inside that function. That's one of the reasons why functions > exist, so that the names you use inside a function are distinct from those > outside. And thank goodness for that! I've been writing Python code since 1997 and version 1.5.,? and I still do a double take when emacs colors all my ids that faint blue that means "builtin." I don't think I've ever used the builtin function id in a program. Ever. Not even once. Honestly, what is a valid use case? That said, I do have boatloads of parameters and objects locally named id because it's idiomatic (at least to me) and mnemonic (at least to me) and just as meaningful. ? No, not continuously. I have eaten and slept since then. From steve+comp.lang.python at pearwood.info Sat Feb 28 01:36:44 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 28 Feb 2015 17:36:44 +1100 Subject: Python Worst Practices References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> Dan Sommers wrote: > On Sat, 28 Feb 2015 12:09:31 +1100, Steven D'Aprano wrote: > >> There's no harm in calling a local variable "id", if you don't use the >> built-in id() inside that function. That's one of the reasons why >> functions exist, so that the names you use inside a function are distinct >> from those outside. > > And thank goodness for that! I've been writing Python code since 1997 > and version 1.5.,? and I still do a double take when emacs > colors all my ids that faint blue that means "builtin." Although it is not helpful for people using screen-readers, and may be of limited use to the colour-blind, I am in favour of colourising built-ins so they stand out. On the other hand, I recall seeing an editor which rejected the idea of colour-coding built-ins, keywords etc., instead it coloured your own variables. So given: spam = 23 eggs += cheese*len(sausage) spam, eggs, cheese and sausage would be different colours. The idea being, when scanning a large code base, all the places that use a specific variable would stand out ("just look for the dark green word"). > I don't think I've ever used the builtin function id in a program. > Ever. Not even once. Honestly, what is a valid use case? Here's one. I think it's the only time I have seen id() used apart from interactive experimentation: https://code.activestate.com/recipes/577504 > That said, I > do have boatloads of parameters and objects locally named id because > it's idiomatic (at least to me) and mnemonic (at least to me) and just > as meaningful. > > ? No, not continuously. I have eaten and slept since then. Slacker! -- Steven From dan at tombstonezero.net Sat Feb 28 02:50:17 2015 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 28 Feb 2015 07:50:17 +0000 (UTC) Subject: Python Worst Practices References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, 28 Feb 2015 17:36:44 +1100, Steven D'Aprano wrote: > Dan Sommers wrote: >> And thank goodness for that! I've been writing Python code since >> 1997 and version 1.5.,? and I still do a double take when >> emacs colors all my ids that faint blue that means "builtin." > Although it is not helpful for people using screen-readers, and may be > of limited use to the colour-blind, I am in favour of colourising > built-ins so they stand out. Now if only emacs were clever enough *not* to colorize "id" when it's one of my names and not the builtin... ;-) > On the other hand, I recall seeing an editor which rejected the idea > of colour-coding built-ins, keywords etc., instead it coloured your > own variables. So given: > > spam = 23 > eggs += cheese*len(sausage) > > spam, eggs, cheese and sausage would be different colours. The idea > being, when scanning a large code base, all the places that use a > specific variable would stand out ("just look for the dark green > word"). As a mostly visual person, I can see (pun intented) the logic and the value in that. I wonder how many variables could be easily distinguished, though, before running out of easily distinguishable colors. Then again, a clever underlying algorithm might choose colors based on *dissimilarity* of the identifiers, so that "i" and "j" would be very diffent colors, but "spam" and "throat_warbler_mangrove" could be the same color because they look so different anyway. >> I don't think I've ever used the builtin function id in a program. >> Ever. Not even once. Honestly, what is a valid use case? > > Here's one. I think it's the only time I have seen id() used apart from > interactive experimentation: > > https://code.activestate.com/recipes/577504 Hah. Very nice. That sort of thing is probably useful for detecting self-referential objects, too (e.g., to prevent infinite output for a circular list). >> ? No, not continuously. I have eaten and slept since then. > > Slacker! Sorry. I'll make up the hours later, I promise! From ethan at stoneleaf.us Sat Feb 28 02:51:49 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 27 Feb 2015 23:51:49 -0800 Subject: Python Worst Practices In-Reply-To: <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54F17395.5060803@stoneleaf.us> On 02/27/2015 10:36 PM, Steven D'Aprano wrote: > Dan Sommers wrote: > >> On Sat, 28 Feb 2015 12:09:31 +1100, Steven D'Aprano wrote: >> >>> There's no harm in calling a local variable "id", if you don't use the >>> built-in id() inside that function. That's one of the reasons why >>> functions exist, so that the names you use inside a function are distinct >>> from those outside. >> >> And thank goodness for that! I've been writing Python code since 1997 >> and version 1.5.,? and I still do a double take when emacs >> colors all my ids that faint blue that means "builtin." > > Although it is not helpful for people using screen-readers, and may be of > limited use to the colour-blind, I am in favour of colourising built-ins so > they stand out. Sure, for the ones I use as built-ins. But I went through the color file for vim and took out the built-ins I use regularly as variables -- and 'id' was the first one to go. -- ~Ethan~ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From ian.g.kelly at gmail.com Sat Feb 28 03:48:44 2015 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 28 Feb 2015 01:48:44 -0700 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Feb 27, 2015 at 9:42 PM, Dan Sommers wrote: > I don't think I've ever used the builtin function id in a program. > Ever. Not even once. Honestly, what is a valid use case? If you have a dict that you want to key on object identity rather than equality, then you can use object ids as the keys. From marko at pacujo.net Sat Feb 28 03:50:19 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 28 Feb 2015 10:50:19 +0200 Subject: Python Worst Practices References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87sidq8lo4.fsf@elektro.pacujo.net> Ethan Furman : > Sure, for the ones I use as built-ins. But I went through the color > file for vim and took out the built-ins I use regularly as variables > -- and 'id' was the first one to go. Ah, yes. The id is with us from the beginning. The self seeks to gratify the whims of the id while at least placating the demands of super. But I think stamping out the id altogether is bound to fail and risk the long-term soundness of the implementation. Marko From rosuav at gmail.com Sat Feb 28 03:55:16 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 19:55:16 +1100 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54f161fd$0$12991$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 28, 2015 at 6:50 PM, Dan Sommers wrote: > Now if only emacs were clever enough *not* to colorize "id" when it's > one of my names and not the builtin... ;-) I think (part of) the point of the colorization is to make it obvious that you've shadowed a builtin. If you use << str = 'spam' >> in your code, the str should be highlighted. ChrisA From rosuav at gmail.com Sat Feb 28 03:58:25 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 19:58:25 +1100 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Feb 28, 2015 at 7:48 PM, Ian Kelly wrote: > On Fri, Feb 27, 2015 at 9:42 PM, Dan Sommers wrote: >> I don't think I've ever used the builtin function id in a program. >> Ever. Not even once. Honestly, what is a valid use case? > > If you have a dict that you want to key on object identity rather than > equality, then you can use object ids as the keys. And not forget to hang onto references to all those objects, else their ids will be meaningless. ChrisA From bc at freeuk.com Sat Feb 28 05:28:33 2015 From: bc at freeuk.com (BartC) Date: Sat, 28 Feb 2015 10:28:33 +0000 Subject: Python Worst Practices In-Reply-To: References: Message-ID: On 25/02/2015 20:58, Michiel Overtoom wrote: > > On Feb 25, 2015, at 21:45, Mark Lawrence wrote: > >> http://www.slideshare.net/pydanny/python-worst-practices I like the way it advises against preserving pixels by removing vowels from identifiers. Then it gives the best practice example of using "color" instead of "clr" or "c"! (Over here it's spelled "colour"...) -- Bartc From bc at freeuk.com Sat Feb 28 05:39:55 2015 From: bc at freeuk.com (BartC) Date: Sat, 28 Feb 2015 10:39:55 +0000 Subject: Python Worst Practices In-Reply-To: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5WgIw.887352$w_7.20106@fx38.am4> On 28/02/2015 01:09, Steven D'Aprano wrote: > Likewise: > > int = 23 > n = int("42") > > Isn't it obvious that the second use of int has to be the built-in function? > I wish that the computer would understand from context which one I mean. (People here would like PL/I then which apparently has *no* reserved words, so you can write: "if if=then then ...") > Other newbie stylistic mistakes which can increase the chance of shadowing > errors include: > > * Too many overly generic variable names like "int" and "str". One thing I find annoying when looking at tutorial examples of an unfamiliar language is when they use identifier names such as "function", "array", "integer", "var" and so on, names which could conceivably be reserved words. Because it's not clear if these *are* keywords that form part of the syntax, or built-ins, or actual made-up user identifiers. -- Bartc From marko at pacujo.net Sat Feb 28 05:56:43 2015 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 28 Feb 2015 12:56:43 +0200 Subject: Python Worst Practices References: Message-ID: <87ioem8ftg.fsf@elektro.pacujo.net> BartC : > (Over here it's spelled "colour"...) The language of science and technology is American English. Learn it like everybody else has to. Marko From denismfmcmahon at gmail.com Sat Feb 28 06:54:01 2015 From: denismfmcmahon at gmail.com (Denis McMahon) Date: Sat, 28 Feb 2015 11:54:01 +0000 (UTC) Subject: requesting you all to please guide me , which tutorials is best to learn redis database References: Message-ID: On Thu, 26 Feb 2015 11:14:50 -0800, Jai wrote: > i want to learn redis database and its use via python , please > guide me which tutorials i should be study, so that i can learn it > in good way Using databases via python often involves working with dictionaries or lists, lists of dictionaries, lists of lists etc. Before you start trying to work with a database and python together, you should have a good grasp of the core python data structures, built in functions, statements, io, statements, flow control etc. You should also have a good grasp of the language (presumably an sql variant) used by the database, and a very good understanding of how string formatting works in python, as you'll be using python to build command strings to send to the database. Then you may be ready to start gluing the two together. -- Denis McMahon, denismfmcmahon at gmail.com From rosuav at gmail.com Sat Feb 28 07:45:05 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Feb 2015 23:45:05 +1100 Subject: Python Worst Practices In-Reply-To: <5WgIw.887352$w_7.20106@fx38.am4> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <5WgIw.887352$w_7.20106@fx38.am4> Message-ID: On Sat, Feb 28, 2015 at 9:39 PM, BartC wrote: > (People here would like PL/I then which apparently has *no* reserved words, > so you can write: "if if=then then ...") Likewise REXX has no reserved words; also, SQL went part-way there, with the notion of "non-reserved keywords". For example, "ORDER" and "BY" are reserved keywords, so they are unavailable as table/column names, but "NULLS" is non-reserved. You could use it, if you wanted to, but it's a syntactic element in some contexts. You can say "ORDER BY some_column NULLS FIRST" or "NULLS LAST" to affect the ordering; you can even say "ORDER BY NULLS NULLS FIRST", which will sort by a column named NULLS, in ascending order, but counting NULL as lower than everything (instead of higher than everything). Given the style of SQL, it'd be insanely restrictive if it had to have everything be either a keyword or nothing, so this is a good half-way-house. ChrisA From wxjmfauth at gmail.com Sat Feb 28 09:26:24 2015 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Sat, 28 Feb 2015 06:26:24 -0800 (PST) Subject: Python Worst Practices In-Reply-To: <87ioem8ftg.fsf@elektro.pacujo.net> References: <87ioem8ftg.fsf@elektro.pacujo.net> Message-ID: Le samedi 28 f?vrier 2015 11:56:57 UTC+1, Marko Rauhamaa a ?crit?: > BartC : > > > (Over here it's spelled "colour"...) > > The language of science and technology is American English. Learn it > like everybody else has to. > Even better, enforce the usage of a language using a script only based on the set of ascii characters. Otherwhile, your beloved (computer) language may crash, as it happened this morning. jmf From breamoreboy at yahoo.co.uk Sat Feb 28 10:33:23 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 28 Feb 2015 15:33:23 +0000 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On 27/02/2015 21:40, Chris Angelico wrote: > On Sat, Feb 28, 2015 at 8:37 AM, Dave Angel wrote: >> Right. In C and C++, instead of being the first slide, it'd be the first 3 >> or 4. Between header file conflicts (especially good because the stdlib >> itself has many multiply-defined symbols, duplicate header files, and >> contradictory include path patterns) > > Yeah, Python has some issues with sys.path and how your local module > can unexpectedly shadow a stdlib one, but at least the stdlib itself > doesn't have any conflicts. I should not ever have to do this dance: > > #include > #undef SOME_SYMBOL > #include > > But sadly, I do. > > ChrisA > As you typed the above up I wonder how many developers around the world were battling with the fun and games caused, particularly when writing cross platform code? It also makes me wonder what idiot decided to use C as the language for the first Python implementation? Or was it written in something else and then ported? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From breamoreboy at yahoo.co.uk Sat Feb 28 10:43:50 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 28 Feb 2015 15:43:50 +0000 Subject: Python Worst Practices In-Reply-To: <87ioem8ftg.fsf@elektro.pacujo.net> References: <87ioem8ftg.fsf@elektro.pacujo.net> Message-ID: On 28/02/2015 10:56, Marko Rauhamaa wrote: > BartC : > >> (Over here it's spelled "colour"...) > > The language of science and technology is American English. Learn it > like everybody else has to. > > > Marko > People from Angleland use any English apart from our own, never. Next thing you'll be telling us to use that new fangled UTC nonsense instead of the clearly correct GMT. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Sat Feb 28 10:46:41 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 1 Mar 2015 02:46:41 +1100 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On Sun, Mar 1, 2015 at 2:33 AM, Mark Lawrence wrote: > > It also makes me wonder what idiot decided to use C as the language for the > first Python implementation? Or was it written in something else and then > ported? Guido, probably. And what other language would you suggest? What other language has comparably extensive multi-platform support? Writing a Python implementation in C instantly makes Python available on all sorts of platforms, with direct access to native libraries on all of them. For example, CPython on Windows can make use of a whole bunch of Microsoft's win32 APIs, via the pywin32 extensions; meanwhile, CPython on Linux can use the inotify functions, again via an extension module (pyinotify or python-inotify). Jython doesn't offer that, as far as I know; or rather, Jython offers access to Java classes rather than to C libraries, and there are a lot more of the latter than the former. Of all the languages that offer convenient access to the same sorts of libraries that C code can (generally, those that compile to machine code and use the same kinds of linker information), which would you suggest as being better than C? C may not be perfect, but it's pretty decent at what it does. ChrisA From cousinstanley at gmail.com Sat Feb 28 11:03:46 2015 From: cousinstanley at gmail.com (Cousin Stanley) Date: Sat, 28 Feb 2015 09:03:46 -0700 Subject: Python Worst Practices References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: > From : Tim Chase > > A quick google-and-tally for languages > and their corresponding number of keywords: > .... re-sorted .... 21 : Lua 31 : Python2.x 33 : Python3.x 33 : C 37 : Pike 40 : Perl 40 : Ruby 50 : Java 54 : Pascal 67 : PHP 77 : C# 86 : C++ -- Stanley C. Kitching Human Being Phoenix, Arizona From rustompmody at gmail.com Sat Feb 28 11:16:03 2015 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 28 Feb 2015 08:16:03 -0800 (PST) Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: <37a2d119-49d5-4bc2-90ea-9fefa54bd645@googlegroups.com> On Saturday, February 28, 2015 at 9:34:05 PM UTC+5:30, Cousin Stanley wrote: > > From : Tim Chase > > > > A quick google-and-tally for languages > > and their corresponding number of keywords: > > .... > > re-sorted .... > > 21 : Lua > 31 : Python2.x > 33 : Python3.x > 33 : C > 37 : Pike > 40 : Perl > 40 : Ruby > 50 : Java > 54 : Pascal > 67 : PHP > 77 : C# > 86 : C++ Ah so after 20 years of development, python has entered the equivalence class of C From breamoreboy at yahoo.co.uk Sat Feb 28 11:28:13 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 28 Feb 2015 16:28:13 +0000 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On 28/02/2015 15:46, Chris Angelico wrote: > On Sun, Mar 1, 2015 at 2:33 AM, Mark Lawrence wrote: >> >> It also makes me wonder what idiot decided to use C as the language for the >> first Python implementation? Or was it written in something else and then >> ported? > > Guido, probably. And what other language would you suggest? What other > language has comparably extensive multi-platform support? Writing a > Python implementation in C instantly makes Python available on all > sorts of platforms, with direct access to native libraries on all of > them. For example, CPython on Windows can make use of a whole bunch of > Microsoft's win32 APIs, via the pywin32 extensions; meanwhile, CPython > on Linux can use the inotify functions, again via an extension module > (pyinotify or python-inotify). Jython doesn't offer that, as far as I > know; or rather, Jython offers access to Java classes rather than to C > libraries, and there are a lot more of the latter than the former. Of > all the languages that offer convenient access to the same sorts of > libraries that C code can (generally, those that compile to machine > code and use the same kinds of linker information), which would you > suggest as being better than C? > > C may not be perfect, but it's pretty decent at what it does. > > ChrisA > I love fishing, just dangle the bait and wait to see what bites :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Sat Feb 28 11:36:37 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 1 Mar 2015 03:36:37 +1100 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On Sun, Mar 1, 2015 at 3:28 AM, Mark Lawrence wrote: > I love fishing, just dangle the bait and wait to see what bites :) Yeah, you usually catch someone. Sometimes they need to be majorly sleep-deprived at half past three in the morning so they're looking at message bodies and skipping the headers that say who actually said stuff... but you'll catch someone. ChrisA From breamoreboy at yahoo.co.uk Sat Feb 28 11:50:22 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 28 Feb 2015 16:50:22 +0000 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On 28/02/2015 16:36, Chris Angelico wrote: > On Sun, Mar 1, 2015 at 3:28 AM, Mark Lawrence wrote: >> I love fishing, just dangle the bait and wait to see what bites :) > > Yeah, you usually catch someone. Sometimes they need to be majorly > sleep-deprived at half past three in the morning so they're looking at > message bodies and skipping the headers that say who actually said > stuff... but you'll catch someone. > > ChrisA > If I had a quid for every time I'd posted when sleep deprived at half past three in the morning, and this on top of Chronic Fatigue Syndrome, then I'd have retired years ago. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From bc at freeuk.com Sat Feb 28 12:55:51 2015 From: bc at freeuk.com (BartC) Date: Sat, 28 Feb 2015 17:55:51 +0000 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On 28/02/2015 15:33, Mark Lawrence wrote: > It also makes me wonder what idiot decided to use C as the language for > the first Python implementation? Or was it written in something else and > then ported? Python was already slow enough even written in C. With any other implementation language, it would have ground to a halt. -- Bartc From python at mrabarnett.plus.com Sat Feb 28 12:56:36 2015 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 28 Feb 2015 17:56:36 +0000 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> Message-ID: <54F20154.3070009@mrabarnett.plus.com> On 2015-02-28 16:03, Cousin Stanley wrote: > >> From : Tim Chase >> >> A quick google-and-tally for languages >> and their corresponding number of keywords: >> .... > > re-sorted .... > > 21 : Lua > 31 : Python2.x > 33 : Python3.x > 33 : C > 37 : Pike > 40 : Perl > 40 : Ruby > 50 : Java > 54 : Pascal > 67 : PHP > 77 : C# > 86 : C++ > Does any language have more than COBOL? That has hundreds! From ethan at stoneleaf.us Sat Feb 28 13:13:01 2015 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 28 Feb 2015 10:13:01 -0800 Subject: Python Worst Practices In-Reply-To: <54F20154.3070009@mrabarnett.plus.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54F20154.3070009@mrabarnett.plus.com> Message-ID: <54F2052D.1010004@stoneleaf.us> On 02/28/2015 09:56 AM, MRAB wrote: > On 2015-02-28 16:03, Cousin Stanley wrote: >> >>> From : Tim Chase >>> >>> A quick google-and-tally for languages >>> and their corresponding number of keywords: >>> .... >> >> re-sorted .... >> >> 21 : Lua >> 31 : Python2.x >> 33 : Python3.x >> 33 : C >> 37 : Pike >> 40 : Perl >> 40 : Ruby >> 50 : Java >> 54 : Pascal >> 67 : PHP >> 77 : C# >> 86 : C++ >> > Does any language have more than COBOL? That has hundreds! I think Visual FoxPro should get at least an (dis)honorable mention -- I didn't count, but there are several page-fulls [1]. -- ~Ethan~ [1] https://msdn.microsoft.com/en-us/library/xztfc506%28v=vs.80%29.aspx -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From python.list at tim.thechases.com Sat Feb 28 13:30:50 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 28 Feb 2015 12:30:50 -0600 Subject: Python Worst Practices In-Reply-To: <54F20154.3070009@mrabarnett.plus.com> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54F20154.3070009@mrabarnett.plus.com> Message-ID: <20150228123050.6be70f3e@bigbox.christie.dr> On 2015-02-28 17:56, MRAB wrote: > On 2015-02-28 16:03, Cousin Stanley wrote: > > > >> From : Tim Chase > >> > >> A quick google-and-tally for languages > >> and their corresponding number of keywords: > >> .... > > > > re-sorted .... > > > > 21 : Lua > > 31 : Python2.x > > 33 : Python3.x > > 33 : C > > 37 : Pike > > 40 : Perl > > 40 : Ruby > > 50 : Java > > 54 : Pascal > > 67 : PHP > > 77 : C# > > 86 : C++ > > > Does any language have more than COBOL? That has hundreds! A quick check using [1] suggests that COBOL has 366, while according to [2], MySQL's flavor of SQL has 825. -tkc [1] http://cobol.404i.com/res.php [2] https://www.drupal.org/node/141051 From breamoreboy at yahoo.co.uk Sat Feb 28 13:38:14 2015 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 28 Feb 2015 18:38:14 +0000 Subject: Python Worst Practices In-Reply-To: References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: On 28/02/2015 17:55, BartC wrote: > On 28/02/2015 15:33, Mark Lawrence wrote: > >> It also makes me wonder what idiot decided to use C as the language for >> the first Python implementation? Or was it written in something else and >> then ported? > > Python was already slow enough even written in C. With any other > implementation language, it would have ground to a halt. > I'm disappointed with the catch so far, this is only the second for my keep net. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python.list at tim.thechases.com Sat Feb 28 13:39:15 2015 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 28 Feb 2015 12:39:15 -0600 Subject: Python Worst Practices In-Reply-To: <54F2052D.1010004@stoneleaf.us> References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <54F20154.3070009@mrabarnett.plus.com> <54F2052D.1010004@stoneleaf.us> Message-ID: <20150228123915.4e540972@bigbox.christie.dr> On 2015-02-28 10:13, Ethan Furman wrote: > On 02/28/2015 09:56 AM, MRAB wrote: > > On 2015-02-28 16:03, Cousin Stanley wrote: > >> > >>> From : Tim Chase > >>> > >>> A quick google-and-tally for languages > >>> and their corresponding number of keywords: > >>> .... > >> > >> re-sorted .... > >> > >> 21 : Lua > >> 31 : Python2.x > >> 33 : Python3.x > >> 33 : C > >> 37 : Pike > >> 40 : Perl > >> 40 : Ruby > >> 50 : Java > >> 54 : Pascal > >> 67 : PHP > >> 77 : C# > >> 86 : C++ > >> > > Does any language have more than COBOL? That has hundreds! > > I think Visual FoxPro should get at least an (dis)honorable mention > -- I didn't count, but there are several page-fulls [1]. That list has ~2405 entries, but it's been long enough since I've used VFP (c. 2000) that I've successfully managed to repress the nuances of whether those are actual reserved words, or whether, like Python built-ins, they're more suggestions rather than enforced. -tkc From gerrit.muller at gmail.com Sat Feb 28 15:51:42 2015 From: gerrit.muller at gmail.com (GerritM) Date: Sat, 28 Feb 2015 12:51:42 -0800 (PST) Subject: PIL installation fails; registration problem Message-ID: <2627328d-f9cb-48ba-bbfb-dfb94d24e001@googlegroups.com> I am reinstalling everything on my new Windows 7 laptop. I run into a problem when installing PIL 1.1.7, in combination with my Activestate Python 2.7.8. The PIL installer complains that no Python is registered. I did run Joakim L?w's script to register Python. This results in the message "*** You probably have another Python installation!". Adding some code to this script reveals that my Python is registered with the following values (deviating from the values in the script): installkey: C:\Python27\ pythonkey C:\Python27\Lib;C:\Python27\DLLs;C:\Python27\Lib\lib-tk The script proposes for pythonkey: C:\Python27;C:\Python27\Lib\;C:\Python27\DLLs\ Can I change the pythonkey to the value proposed by Joakim L?w's script? From yves at zioup.com Sat Feb 28 18:12:54 2015 From: yves at zioup.com (yves at zioup.com) Date: Sat, 28 Feb 2015 16:12:54 -0700 Subject: suggestions for functional style (singleton pattern?) Message-ID: Hi, For some scripts, I write in a a more functional way, using a lot of small functions outside of any class. Although it makes the code clearer for specific cases, I have found that it makes debugging and using the repl in general difficult, as as I have to re-initialise every single objects every time. I have now started to use some kind of state pattern to alleviate this, here's a simplistic example: https://github.com/dorfsmay/state_pattern_for_debugging_python/blob/master/dirstats.py Are there better ways to address this? Any suggestion on this style? Thanks. -- http://yves.zioup.com gpg: 4096R/32B0F416 From greg.ewing at canterbury.ac.nz Sat Feb 28 19:29:46 2015 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 01 Mar 2015 13:29:46 +1300 Subject: Python Worst Practices In-Reply-To: References: <54f1154c$0$12985$c3e8da3$5496439d@news.astraweb.com> <5WgIw.887352$w_7.20106@fx38.am4> Message-ID: Chris Angelico wrote: > Likewise REXX has no reserved words; also, SQL went part-way there, > with the notion of "non-reserved keywords". Python sometimes has those, too. For example, the "as" in "import as" was non-reserved when it was first introduced, to avoid abruptly breaking code that used it as a name. -- Greg From steve+comp.lang.python at pearwood.info Sat Feb 28 21:08:58 2015 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 01 Mar 2015 13:08:58 +1100 Subject: Python Worst Practices References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> Message-ID: <54f274bb$0$13011$c3e8da3$5496439d@news.astraweb.com> BartC wrote: > On 28/02/2015 15:33, Mark Lawrence wrote: > >> It also makes me wonder what idiot decided to use C as the language for >> the first Python implementation? Or was it written in something else and >> then ported? > > Python was already slow enough even written in C. Tell that to the IronPython developers. Of course, benchmarks are very variable, depending on what you wish to do, and a few things (like exceptions) are very much slower in IronPython: http://ironpython.codeplex.com/wikipage?title=IP27A1VsCPy27Perf&referringTitle=IronPython%20Performance but overall, I think it's fair to say that IronPython is about 1.8 times the speed of CPython. > With any other implementation language, it would have ground to a halt. That's laughably inaccurate. -- Steven From torriem at gmail.com Sat Feb 28 21:19:32 2015 From: torriem at gmail.com (Michael Torrie) Date: Sat, 28 Feb 2015 19:19:32 -0700 Subject: suggestions for functional style (singleton pattern?) In-Reply-To: References: Message-ID: <54F27734.8030101@gmail.com> On 02/28/2015 04:12 PM, yves at zioup.com wrote: > For some scripts, I write in a a more functional way, using a lot of small > functions outside of any class. Although it makes the code clearer for > specific cases, I have found that it makes debugging and using the repl in > general difficult, as as I have to re-initialise every single objects every time. > > I have now started to use some kind of state pattern to alleviate this, here's > a simplistic example: > > https://github.com/dorfsmay/state_pattern_for_debugging_python/blob/master/dirstats.py > > Are there better ways to address this? Any suggestion on this style? You say you are trying to use a singleton pattern, but your code does not appear to implement a singleton. From what I can read of your code, you really should just put all your functions as methods to the DirStat class and call it good. Especially if your module is going to be used in several places potentially at the same time. If what you want is a singleton, then that's what a module is already. Module functions are singleton methods and module attributes maintain state. Just call your module dirstat and interface with it: dirstat.opendir(**vars(args) dirstat.print() In this example, I really don't think the singleton is a good pattern, though. From rosuav at gmail.com Sat Feb 28 21:43:50 2015 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 1 Mar 2015 13:43:50 +1100 Subject: Python Worst Practices In-Reply-To: <54f274bb$0$13011$c3e8da3$5496439d@news.astraweb.com> References: <7053A277-9687-49B0-9FDB-CB4DB3E76DEC@gmail.com> <54F0E38D.40006@davea.name> <54f274bb$0$13011$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Mar 1, 2015 at 1:08 PM, Steven D'Aprano wrote: > but overall, I think it's fair to say that IronPython is about 1.8 times the > speed of CPython. > > >> With any other implementation language, it would have ground to a halt. > > That's laughably inaccurate. I'm sure that Python could have been implemented in FORTRAN without losing performance. Conversely, I'm sure Python could also have been implemented on top of BASIC if someone felt like it, though what the advantages might be I have no idea. But performance is not (or should not be) the primary reason for choosing a language. Imagine if all your Python code ran twice as fast (that's slightly better than the IronPython figure quoted!), but worked only on BSD Unix and Mac OS. Is that something that'll make a fledgling language succeed? Or would universality, even at the cost of performance, lead to a greater userbase and development team, which ultimately would result in far greater improvements in both functionality and performance? ChrisA From marfig at gmail.com Sat Feb 28 22:45:54 2015 From: marfig at gmail.com (Mario Figueiredo) Date: Sun, 01 Mar 2015 04:45:54 +0100 Subject: suggestions for functional style (singleton pattern?) References: Message-ID: <13v4falb4odnhtvss4qdatnn16sgiv5pgd@4ax.com> On Sat, 28 Feb 2015 16:12:54 -0700, yves at zioup.com wrote: >Hi, > >For some scripts, I write in a a more functional way, using a lot of small >functions outside of any class. Although it makes the code clearer for >specific cases, I have found that it makes debugging and using the repl in >general difficult, as as I have to re-initialise every single objects every time. > >I have now started to use some kind of state pattern to alleviate this, here's >a simplistic example: > >https://github.com/dorfsmay/state_pattern_for_debugging_python/blob/master/dirstats.py > >Are there better ways to address this? Any suggestion on this style? (warning: Didn't bother prove-reading and spell chocking. Sorry...) Problem 1: With the exception of get_all_files(), all your functions do is to call another standard function. This decreases performance unnecessarily and may mislead the users of your API into thinking these functions perform some different type of average or min/max operations. For a single call to your API, the performance issue isn't a problem. But if some user wraps a DirStat instance in a loop, they may suffer an unnecessary hit. Meanwhile, If I see a calculate_average() or a find_smallest() function in your API, I immediately deduce it must be doing something different than a standard average or min/max operation. Otherwise why would the API code bother about something I can easily do myself? And when I go look in your code trying to understand why you are offering me an average function, I'll become dissapointed. Solution: Do not create an interface for standard functions, unless you plan to change how these functions work. Your desire to let your users know that they can calculate an average from your API object will fall in deaf ears because they can realize that from themselves. An API should only expose unique methods or complex operations wrapped in one or two methods. If you are coding a method with a single line and that line starts with return and is followed by a function call, you know you are creating a redundant interface. Problem 2: No one likes long names. And if some user of your API is coding under an 80 character per line limit, they'll hate you for it. Your functions contain redundant terms that only increase the function name size and do not offer any advantage in having a better grasp of what the function does. average() is better than calculate_average(), for instance. Solution: Always be careful with how you name your public interface elements. These will define much of the 'personality' of your API. calculate_average() makes for a tacky API. average() makes for a pretty standard API. Problem 3: Modules are self-contained contextualized units that must contain only code that fits in the namespace the module defines for itself. If this module is called dirstats and is chiefed by a DirStats object that manages a dictionary of files, a public calculate_average() function or a min/max pair of functions simply don't fit within the module context. At the very least they should be moved inside the class so it becomes clear to anyone caring that these functions really only matter in the context of the DirStats object. (But because of problem 1, these functions should not even exist and instead be replaced by direct calls to the statistics module functions inside the DirStats class.) The get_all_files() function is an offender. It should be moved inside the class and probably even be removed entirely. Since it apparently has no other objective than to help the class __init__() method, the function code just just be copied/pasted inside that method. Solution: Do not make a module a house of confusion. Each module should contain only public functions and objects that belong with each other under the module proposed context. Also look carefully at what you are putting outside of a class. You are making it a part of the public interface for that module. The only reason to do so is if it can have some use outside the class definition. Problem 4: You speak of a singleton. But you haven't implemented one. It is not clear from your code if this class should be a singleton. I'm guessing not. Singletons are in fact rare. Well, let me put it another way: Good reasons to code a singleton are rare. A good singleton, for instance could be the Map() object in a strategy game, or the canonical Logging() class in most programs that implement logging. These are really one-time objects that make sense to ever always exist as just one instance. Solution: But, if you ever wish to implement a singleton, the following pattern mostly works and is short and to the point: class Map(): _instance = None def __new__(cls, *args, **kwargs): if Map._instance is None: Map._instance = super(Map, cls).__new__(cls) return Map._instance >>> a = Map() >>> b = Map() >>> a is b True Just remember, for most singletons the truth is that what you think you will only need 1 today, you will need 2 tomorrow. Problem 5: A pet peeve of mine. If you aren't going to use a variable, make that explicit in your code by taking advatange of Python wonderful language features. In your get_all_files() function, 'dirs' is not used. So make that explicit to you and anyone else reading your code, by using either the underscore or double underscore variable naming convention. Solution: It goes like this: for root, __, files in os.walk(directory): for fn in files: abs_path = os.path.join(root, fn) all_files[abs_path] = os.path.getsize(abs_path) .... So to finalize. How should your dirstats.py module probably look like? Something like this: import os import inspect import argparse import operator import statistics class DirStat(object): def __init__(self, directory): self.all_files = dict() for root, dirs, files in os.walk(directory): for fn in files: abs_path = os.path.join(root, fn) self.all_files[abs_path] = os.path.getsize(abs_path) self.average = statistics.mean(self.all_files.values()) self.median = statistics.median(self.all_files.values()) self.largest = max(self.all_files, self.all_files.get) self.smallest = min(self.all_files, self.all_files.get) def print(self): print("mean: {:.2f}".format(self.average)) print("median: {:.2f}".format(self.median)) print("smallest file: {}".format(self.smallest)) print("largest file: {}".format(self.largest)) def __repr__(self): return self.__dict__.__repr__() I also wanted to say something about your __repr__, but this post is already too long. From yves at zioup.com Sat Feb 28 23:11:00 2015 From: yves at zioup.com (yves at zioup.com) Date: Sat, 28 Feb 2015 21:11:00 -0700 Subject: suggestions for functional style (singleton pattern?) In-Reply-To: References: Message-ID: On 2015-02-28 19:19, Michael Torrie wrote: > You say you are trying to use a singleton pattern, but your code does > not appear to implement a singleton. From what I can read of your code, I call it a singletone because I only every create one object. I am not trying to use a singleton, I'm trying to avoid issues created by the alternative ways of doing this (not having access to the variables in the repl etc...). > you really should just put all your functions as methods to the DirStat > class and call it good. Especially if your module is going to be used > in several places potentially at the same time. There are type of problems where I don't want to bind functions to data. By the way, I have just added some more explanation in the code, trying to clarify the problem. -- http://yves.zioup.com gpg: 4096R/32B0F416 From yves at zioup.com Sat Feb 28 23:29:03 2015 From: yves at zioup.com (yves at zioup.com) Date: Sat, 28 Feb 2015 21:29:03 -0700 Subject: suggestions for functional style (singleton pattern?) In-Reply-To: <13v4falb4odnhtvss4qdatnn16sgiv5pgd@4ax.com> References: <13v4falb4odnhtvss4qdatnn16sgiv5pgd@4ax.com> Message-ID: On 2015-02-28 20:45, Mario Figueiredo wrote: > Problem 1: > With the exception of get_all_files(), all your functions do is to > call another standard function. This decreases performance > unnecessarily and may mislead the users of your API into thinking > these functions perform some different type of average or min/max > operations. I tried to create a simple example to make a point, I wouldn't create function wrappers like this. I'll to think of a better example, this was to get the discussion going. > Problem 3: > At the very least they should be moved inside the class so it becomes > clear to anyone caring that these functions really only matter in the > context of the DirStats object. Trying to write in a more functional way, not OO here, the functions could be used by more than one class. I understand OO, use it a lot, but sometimes try to move beyond that. > Problem 4: > You speak of a singleton. But you haven't implemented one. It is not > clear from your code if this class should be a singleton. I'm guessing > not. Singletons are in fact rare. Well, let me put it another way: > Good reasons to code a singleton are rare. Thanks. I hadn't realise "singleton" meant a class built such that it could not be instanciated more than once, I thought it corresponded to a pattern where only one object is ever created from a given class. > But, if you ever wish to implement a singleton, the following pattern > mostly works and is short and to the point: > > class Map(): > _instance = None > > def __new__(cls, *args, **kwargs): > if Map._instance is None: > Map._instance = super(Map, cls).__new__(cls) > return Map._instance > > >>> a = Map() > >>> b = Map() > >>> a is b > True Thanks. > Problem 5: > A pet peeve of mine. If you aren't going to use a variable, make that > explicit in your code by taking advatange of Python wonderful language > features. Good point - thanks. -- http://yves.zioup.com gpg: 4096R/32B0F416